-- Logs begin at Wed 2022-11-02 06:44:56 UTC, end at Wed 2022-11-02 07:54:50 UTC. -- Nov 02 07:14:10.330435 dv-cinder-778193-30-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Nov 02 07:14:10.343057 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Nov 2 07:14:10 2022] *** Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: nodename: dv-cinder-778193-30-iscsi Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: machine: x86_64 Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: clock source: unix Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: pcre jit disabled Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: detected number of CPU cores: 4 Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: current working directory: / Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: detected binary path: /usr/bin/uwsgi-core Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: your processes number limit is 47680 Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: your memory page size is 4096 bytes Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: detected max file descriptor number: 2048 Nov 02 07:14:10.344811 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: lock engine: pthread robust mutexes Nov 02 07:14:10.345652 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: thunder lock: enabled Nov 02 07:14:10.345652 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Nov 02 07:14:10.345652 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Nov 02 07:14:10.384039 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: Python main interpreter initialized at 0x55f3d94d09c0 Nov 02 07:14:10.384039 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: python threads support enabled Nov 02 07:14:10.384039 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: your server socket listen backlog is limited to 100 connections Nov 02 07:14:10.384039 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: your mercy for graceful operations on workers is 90 seconds Nov 02 07:14:10.384374 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: mapped 403077 bytes (393 KB) for 2 cores Nov 02 07:14:10.384452 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: *** Operational MODE: preforking *** Nov 02 07:14:10.384709 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: *** uWSGI is running in multiple interpreter mode *** Nov 02 07:14:10.384709 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: spawned uWSGI master process (pid: 81489) Nov 02 07:14:10.384905 dv-cinder-778193-30-iscsi systemd[1]: Started Devstack devstack@keystone.service. Nov 02 07:14:10.386472 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: spawned uWSGI worker 1 (pid: 81502, cores: 1) Nov 02 07:14:10.387169 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: spawned uWSGI worker 2 (pid: 81503, cores: 1) Nov 02 07:14:10.387473 dv-cinder-778193-30-iscsi devstack@keystone.service[81489]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Nov 02 07:14:14.591770 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:14:14.591770 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: from cryptography.utils import int_from_bytes Nov 02 07:14:14.592611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:14:14.592611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Nov 02 07:14:14.593027 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:14:14.593027 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: from cryptography.utils import int_from_bytes Nov 02 07:14:14.593132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:14:14.593132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Nov 02 07:14:14.641066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 07:14:14.641257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 07:14:14.641433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] command line args: [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 07:14:14.641666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 07:14:14.641892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ================================================================================ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 07:14:14.642079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] admin_token = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.642379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_dir = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.642509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.642728 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_source = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.642915 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] debug = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.643087 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.643474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.643603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.643881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.644086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.644345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.644554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_config_append = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.644648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.644842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_dir = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.645002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.645259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.645508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.645695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.645988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.646151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.646296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.646537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.646773 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.646960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 07:14:14.647139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.647308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 07:14:14.647453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.647621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] command line args: [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 07:14:14.647780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.647888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 07:14:14.648126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.648291 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ================================================================================ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 07:14:14.648451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.648594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.648761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.648930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_dir = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.649033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.649273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.649273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_source = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.649273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] debug = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.649695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.649841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.650035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] publish_errors = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.650236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.650388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.650531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.650671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.650816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.650930 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.651045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.651165 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.651313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.651727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.652246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_config_append = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.652372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] standard_threads = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.652492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.652605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.652724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_dir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.652724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.652932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.652932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.653150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.653277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_journal = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.653390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.653514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_json = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.653664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.653781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_stderr = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.653781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_syslog = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.653995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.654127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.654239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.654239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.654461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.654571 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.654691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.654801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.654801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.655047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.655168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.655285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.655402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.655512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.655627 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.655737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.655858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.655972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.656087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.656228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.656370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.656498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.656498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.656781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.656893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.657001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] publish_errors = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.657121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.657233 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.657348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.657469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.657580 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.657688 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.657805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.657915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.658024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.658159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.658277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.658389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] standard_threads = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.658512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.658631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.658748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.658858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.658858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.659067 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.659185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_journal = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.659303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.659419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_json = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.659537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.659656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_stderr = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.659656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_syslog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.659875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.659988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:14.660097 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.660205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.660316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.660425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.660544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.660662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.660785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.660897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.661981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.662210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.662332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.662443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.662561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.662681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.662800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.662911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.663955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.664972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.665973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.666083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.666218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.666327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.666437 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.666556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.666675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.666786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.666896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.667050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.667597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.667730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.667931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.668955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.669983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.670939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671733 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.671954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672371 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.672975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673619 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.673962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.674942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675390 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.675944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.676062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.676181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.676298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.676410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.676519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.676700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.676813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.676923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.external = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.677836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.password = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.token = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.678954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679282 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679728 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.679955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.680094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.680217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.680329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.680447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.680561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.680683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.680795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.680905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681688 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.681918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.682034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.682161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.682283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.682396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.682516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.682698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.683179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.683299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.683299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.external = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.683510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.683631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.683746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.683746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.683954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.password = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.token = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684619 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684619 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.684948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.685974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.686127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.686243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.686367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.686481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.686597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.686710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.686822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.686933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687164 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.687956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.688972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.689963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.690101 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.690225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.690414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.690529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.690640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.690751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.690861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.690972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.691991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.692102 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.692213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.692323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.692442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.692555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.692673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.692786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.692896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.693913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694629 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.694968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.695078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.695201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.695314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.695424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.695534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.695651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.695768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.695888 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.696971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697193 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.697997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.698122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.698273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.698491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.698931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.699160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.699324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.699441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.699551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.699661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.699771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.699890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.700677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.701188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.701300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.701300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.701300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.701300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.701707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.701825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.701937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.702985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703435 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703435 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.703981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.704480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.704593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.704704 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.704814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.704814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.705905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.706125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.706238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.706349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.706469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.706582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.706766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.706880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.707037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.707226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.707439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.707556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.707667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.707894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.708006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.708117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.708345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.708459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.708590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.708823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.708942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.709053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.709273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.709397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.709508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.709744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.709857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.709968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.710239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.710357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.710467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.710696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.710809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.710940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.711141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.711257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.711405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.711595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.711803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.712347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.712462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.712573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.712787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.712901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.713046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.713249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.713374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.713506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.713711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.713919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.714183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.714363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.714512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.714650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.714806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.714936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.715048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.715238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.715355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.715466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.715561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.715716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.715833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.715957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716293 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.716965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717528 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.717977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.718110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.718223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.718335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.718445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.718584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.718713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.718826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.718938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.719049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.719159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.719271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.719391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.719524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.719639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.719772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.719891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.720928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721041 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.721967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.722077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.722243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.722366 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.722481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.722592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.722703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.722814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.722925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723724 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.723949 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.724956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 07:14:14.725074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.725186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.725334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.725582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.725828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.726074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.726345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.726753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.726992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.727241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.727489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.727735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.727988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.728220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.728472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.728723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.728962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.729200 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.729430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.729705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.729948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.730230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.730478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.730714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.730950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.731181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.731415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.731650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.731897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.732145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.732402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.732651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.732875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.733124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.733365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.733613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.733848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.734103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.734340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.734584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.734864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.735095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.735337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.735603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.735863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.736107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.736353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.736589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.736827 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.737060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.737293 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.737526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.737764 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.738006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.738266 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.738509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.738741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.738987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.739220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.739462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.739690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.739929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.740161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:14.740329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 07:14:14.753946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.754202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.754427 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.762799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.765565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.765800 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.766025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.774336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.791647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.791879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.793702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.793932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.843551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.843898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.891442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.891648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.892595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:14.892828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.047462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.047671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.047821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.047961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.048065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.048132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.048262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.048344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.048455 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.048535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.048657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.048882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.072105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.072323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.072604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.072768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.072864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.072931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.072991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.073162 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.073255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.073428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.073510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.073638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.073873 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.074191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.084896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.085374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.098191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.098387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.098588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.098702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.098776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.098842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.099011 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.099135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.099277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.099362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.099424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.099488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.099635 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.099852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:15.116039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.116249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:14:15.116588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.116790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.116904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:14:15.116990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:14:15.117058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.117299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.117421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.117496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:14:15.117746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.117869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.117979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.118066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.118556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.118817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.127633 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.127911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.128179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.128302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.128482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.128618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.128772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.129053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.135442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.136235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.138458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.138763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.138931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.139026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.139238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.139364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.139519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.139646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.139760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.139846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.140071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.140209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.140346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.140427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.140544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.140718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.140857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.140994 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.141163 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.141251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.141471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.141741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.163885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.164179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.165334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.165614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.165739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.165873 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.172072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.172291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.172434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.172553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.172655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.172853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.178996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.179203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.179302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.179489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.183424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.183577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.183704 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.183911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.184034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.184127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.184250 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.184459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.184582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.184689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.184789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.184992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.185116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.185261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.185385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.185500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.200833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.200959 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.201124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.201220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.204979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.205123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.208232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.208358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.208524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.208649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.213066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.213191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.216247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.216371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.219474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.219619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.219772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.219896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.220064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.220189 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.220326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.220414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.220647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.220772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.220922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.221060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.236621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.236746 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.236899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.237023 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.237144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.237253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.237419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.237544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.244253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.244378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.245817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.245999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.247393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.247517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.247678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.247803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.250312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.250516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.250636 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.250799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.250923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.251186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.251310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.251464 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.251603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.251758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.262544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.262837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.262960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.263178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.263301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.263475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.263598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.263771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.263896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.264057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.264196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.264303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.264389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.264634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.279895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.280045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.282790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.282916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.285658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.285784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.287095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.287220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.288687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.288812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.288962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.289087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.289222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.289314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.295666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.295790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.297139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.297283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.300146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.300271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.300432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.300557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.300717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.300861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.301017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.301091 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.308916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.309042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.309211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.309335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.309484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.309609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.317195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.317343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:15.317447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.317570 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.317752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.317876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.317992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.318141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.318310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.318434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.318551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.318641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.318850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.318989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.319184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.319310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.319421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.319510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.319689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.319812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.319983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.320090 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.320226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.320317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.320495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.320619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.320736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.320840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:15.353917 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f3d94d09c0 pid: 81503 (default app) Nov 02 07:14:15.354037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f3d94d09c0 pid: 81502 (default app) Nov 02 07:14:15.357350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a002b0-b62e-4497-a054-8f576b3759de None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.357635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a002b0-b62e-4497-a054-8f576b3759de None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.357912 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a002b0-b62e-4497-a054-8f576b3759de None None] PATH_INFO: `/v3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.362085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:22.133321 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450961fc-2ce0-4d96-97e6-253f5945ff2d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.134198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450961fc-2ce0-4d96-97e6-253f5945ff2d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.134198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450961fc-2ce0-4d96-97e6-253f5945ff2d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.141284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:22 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:22.147817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.148300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.148438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.219497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: INFO dbcounter [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] Registered counter for database keystone Nov 02 07:14:22.240986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG oslo_db.sqlalchemy.engines [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81503) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 07:14:22.252591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writer thread running {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 07:14:22.436245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81503) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 07:14:22.436661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81503) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 07:14:22.495886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:22.497429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:22.640615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:14:23.115700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450961fc-2ce0-4d96-97e6-253f5945ff2d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:23.186657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: INFO dbcounter [None req-450961fc-2ce0-4d96-97e6-253f5945ff2d None None] Registered counter for database keystone Nov 02 07:14:23.211736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG oslo_db.sqlalchemy.engines [None req-450961fc-2ce0-4d96-97e6-253f5945ff2d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 07:14:23.224146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writer thread running {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 07:14:23.346533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450961fc-2ce0-4d96-97e6-253f5945ff2d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:23.351847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c434a-5f65-49c9-9429-ea78358800a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:23.352466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c434a-5f65-49c9-9429-ea78358800a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:23.352740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c434a-5f65-49c9-9429-ea78358800a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:23.353016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c434a-5f65-49c9-9429-ea78358800a2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:23.362326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c434a-5f65-49c9-9429-ea78358800a2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:23.412729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d80c434a-5f65-49c9-9429-ea78358800a2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 07:14:23.413068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c434a-5f65-49c9-9429-ea78358800a2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:23.423057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Wed Nov 2 07:14:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:14:23.433320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:23.452508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc8fdd-cbe4-4c93-aed6-9392cb40e1c2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:23.458798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19632f1d-4d29-4781-afbe-13783f5f05c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:23.459921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632f1d-4d29-4781-afbe-13783f5f05c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:23.460204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632f1d-4d29-4781-afbe-13783f5f05c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:23.460473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632f1d-4d29-4781-afbe-13783f5f05c6 admin admin] PATH_INFO: `/v3/projects/e25fb0b05a884f4d989636700db0537f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:23.465185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19632f1d-4d29-4781-afbe-13783f5f05c6 admin admin] RBAC: Authorizing `identity:get_project(project_id=e25fb0b05a884f4d989636700db0537f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:23.516511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING keystone.common.rbac_enforcer.enforcer [None req-19632f1d-4d29-4781-afbe-13783f5f05c6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 07:14:23.516848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19632f1d-4d29-4781-afbe-13783f5f05c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:23.521332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Wed Nov 2 07:14:23 2022] GET /identity/v3/projects/e25fb0b05a884f4d989636700db0537f => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:25.632013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2b8de-a393-4caf-bc61-3875bcc80acf None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.632988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2b8de-a393-4caf-bc61-3875bcc80acf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.632988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2b8de-a393-4caf-bc61-3875bcc80acf None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.639626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:25.648114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c115-4b27-4edf-9cdc-f2fb2988d640 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.648569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c115-4b27-4edf-9cdc-f2fb2988d640 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.648658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c115-4b27-4edf-9cdc-f2fb2988d640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.670579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a801c115-4b27-4edf-9cdc-f2fb2988d640 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:25.672068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a801c115-4b27-4edf-9cdc-f2fb2988d640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:25.681173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:14:25.947170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2b8de-a393-4caf-bc61-3875bcc80acf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:25.963182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2b8de-a393-4caf-bc61-3875bcc80acf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:25.968437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9947fd-495b-4bae-8e57-7e67cd70216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:25.969040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9947fd-495b-4bae-8e57-7e67cd70216c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.969322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9947fd-495b-4bae-8e57-7e67cd70216c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.969595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9947fd-495b-4bae-8e57-7e67cd70216c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.977415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9947fd-495b-4bae-8e57-7e67cd70216c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:25.979255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9947fd-495b-4bae-8e57-7e67cd70216c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:25.988773 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Wed Nov 2 07:14:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:14:25.997572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801c115-4b27-4edf-9cdc-f2fb2988d640 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:26.012671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801c115-4b27-4edf-9cdc-f2fb2988d640 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:26.019014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202ce85-3923-4466-bddf-fd09e950f481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:26.019636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ce85-3923-4466-bddf-fd09e950f481 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.019911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ce85-3923-4466-bddf-fd09e950f481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.020176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ce85-3923-4466-bddf-fd09e950f481 admin admin] PATH_INFO: `/v3/users/085301de455d4179ae28198d5b0bd385` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.021716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202ce85-3923-4466-bddf-fd09e950f481 admin admin] RBAC: Authorizing `identity:get_user(user_id=085301de455d4179ae28198d5b0bd385)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:26.023632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202ce85-3923-4466-bddf-fd09e950f481 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:26.025141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Wed Nov 2 07:14:25 2022] GET /identity/v3/users/085301de455d4179ae28198d5b0bd385 => generated 247 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:28.191288 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f56335-b803-45d9-98f7-d915f7ba56f7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.191801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f56335-b803-45d9-98f7-d915f7ba56f7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.191801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f56335-b803-45d9-98f7-d915f7ba56f7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.193077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:28.198434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5149-cac7-47da-885d-b5850d6a11f4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.198852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5149-cac7-47da-885d-b5850d6a11f4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.199024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5149-cac7-47da-885d-b5850d6a11f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.217924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-503a5149-cac7-47da-885d-b5850d6a11f4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:28.219177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-503a5149-cac7-47da-885d-b5850d6a11f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:28.237918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaacb46-9cd4-47af-b661-d95ec9e306da None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.238293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaacb46-9cd4-47af-b661-d95ec9e306da None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.238573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaacb46-9cd4-47af-b661-d95ec9e306da None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.239908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:28.245021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.245817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.246119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.289465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG passlib.handlers.bcrypt [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81502) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 07:14:28.289945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG passlib.handlers.bcrypt [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81502) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 07:14:28.303808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:28.338851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:28.340602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:28.441785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:28.565649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a5149-cac7-47da-885d-b5850d6a11f4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:28.591305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:28.674189 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a5149-cac7-47da-885d-b5850d6a11f4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:28.703414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:28.709463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9a5f-f14c-469e-8c31-dd69e1047a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:28.710121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9a5f-f14c-469e-8c31-dd69e1047a42 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.710413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9a5f-f14c-469e-8c31-dd69e1047a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.710689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9a5f-f14c-469e-8c31-dd69e1047a42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.722392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b9a5f-f14c-469e-8c31-dd69e1047a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-517b9a5f-f14c-469e-8c31-dd69e1047a42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:28.732546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:28.733631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:28.734658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:14:28.735585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:28.742287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9a5f-f14c-469e-8c31-dd69e1047a42 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:28.809639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fef055-e25d-47a4-9dd6-9b7597800ed4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:28.811984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:28.825784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:28.845747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9a5f-f14c-469e-8c31-dd69e1047a42 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:28.850317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:28.851181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.851454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.851719 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.852690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:28.854404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:28.866254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.sql.core [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:14:28.866254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:14:28.866254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [parameters: {'id': '0c1295dbbd604070909592b3ef655ef2', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:14:28.866254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:14:28.879413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:28.880017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.880292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.880559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.881221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:28.881519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:28.882962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:14:28.884264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:14:28.885005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application yield resource Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise self.value Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application session.commit() Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:14:28.886137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:14:28.887052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:14:28.887908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:14:28.888649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [parameters: {'id': '0c1295dbbd604070909592b3ef655ef2', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:14:28.889458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:28.890232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:28.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 8/15] 10.222.0.25 () {60 vars in 1137 bytes} [Wed Nov 2 07:14:28 2022] POST /identity/v3/domains => generated 190 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:28.891775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:28.897251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 8/16] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:28.903157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:28.997877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ef6f1-9acc-429b-b927-4d2ab9d2d064 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:29.006904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d81a6c-6257-4cc1-9715-57902e4860e0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:29.031049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:29.031919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:29.032250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:29.032532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:29.040946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee3197-dfad-41e4-9b23-b5b8dfb89ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:29.041577 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee3197-dfad-41e4-9b23-b5b8dfb89ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:29.041904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee3197-dfad-41e4-9b23-b5b8dfb89ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:29.042202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee3197-dfad-41e4-9b23-b5b8dfb89ec7 None admin] PATH_INFO: `/v3/users/085301de455d4179ae28198d5b0bd385` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:29.043361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee3197-dfad-41e4-9b23-b5b8dfb89ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=085301de455d4179ae28198d5b0bd385)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:29.044820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee3197-dfad-41e4-9b23-b5b8dfb89ec7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:29.046404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:28 2022] GET /identity/v3/users/085301de455d4179ae28198d5b0bd385 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:29.050479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:29.051941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:29.052045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee3197-dfad-41e4-9b23-b5b8dfb89ec7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:29.066973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:29.067964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:29.068925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:29.069718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:29.069718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:29.069718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:29.069718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:29.075511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:29.100579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:29.155562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee3197-dfad-41e4-9b23-b5b8dfb89ec7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:29.178427 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa7b79-6aff-4685-904a-5037de3e747f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:29.188540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48dbc4-a19d-4762-bf2d-d4c42fca8854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:29.189168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48dbc4-a19d-4762-bf2d-d4c42fca8854 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:29.189445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48dbc4-a19d-4762-bf2d-d4c42fca8854 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:29.189717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48dbc4-a19d-4762-bf2d-d4c42fca8854 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:29.190859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48dbc4-a19d-4762-bf2d-d4c42fca8854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:29.192321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48dbc4-a19d-4762-bf2d-d4c42fca8854 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:29.193857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:29.199847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48dbc4-a19d-4762-bf2d-d4c42fca8854 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:29.211377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e04832-0e6f-439a-8962-f144ee6cb17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:29.212269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e04832-0e6f-439a-8962-f144ee6cb17b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:29.212542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e04832-0e6f-439a-8962-f144ee6cb17b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:29.212813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e04832-0e6f-439a-8962-f144ee6cb17b None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:29.213452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e04832-0e6f-439a-8962-f144ee6cb17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:29.214082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e04832-0e6f-439a-8962-f144ee6cb17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:29.215605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e04832-0e6f-439a-8962-f144ee6cb17b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:29.233025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 10/20] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:29.287902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:29.310591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48dbc4-a19d-4762-bf2d-d4c42fca8854 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:29.349609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54053b62-3bf4-4bc4-a7b6-7ccb5e5ff6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:29.350305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54053b62-3bf4-4bc4-a7b6-7ccb5e5ff6d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:29.350585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54053b62-3bf4-4bc4-a7b6-7ccb5e5ff6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:29.350862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54053b62-3bf4-4bc4-a7b6-7ccb5e5ff6d3 None admin] PATH_INFO: `/v3/domains/default/users/085301de455d4179ae28198d5b0bd385/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:29.352957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54053b62-3bf4-4bc4-a7b6-7ccb5e5ff6d3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=085301de455d4179ae28198d5b0bd385, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:29.355204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54053b62-3bf4-4bc4-a7b6-7ccb5e5ff6d3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:29.474063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 11/21] 10.222.0.25 () {58 vars in 1361 bytes} [Wed Nov 2 07:14:29 2022] PUT /identity/v3/domains/default/users/085301de455d4179ae28198d5b0bd385/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:31.891820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac6a8d-67a5-4e88-b544-61c339775bb1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.892337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac6a8d-67a5-4e88-b544-61c339775bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.892337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac6a8d-67a5-4e88-b544-61c339775bb1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.893701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 11/22] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:31.905294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27e23c-eefe-4c9c-9701-e12ee9fc01b5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.905794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27e23c-eefe-4c9c-9701-e12ee9fc01b5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.905868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27e23c-eefe-4c9c-9701-e12ee9fc01b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.939956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9b27e23c-eefe-4c9c-9701-e12ee9fc01b5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:31.941478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9b27e23c-eefe-4c9c-9701-e12ee9fc01b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:32.035228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:32.207413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4e46f-7aa9-46b0-8603-7ee2d5d5f7b6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.207704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4e46f-7aa9-46b0-8603-7ee2d5d5f7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.207880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4e46f-7aa9-46b0-8603-7ee2d5d5f7b6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.209196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 12/24] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:32.228288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b391f-0e04-41b0-8f87-a1e1bef565e8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.228616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b391f-0e04-41b0-8f87-a1e1bef565e8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.228850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b391f-0e04-41b0-8f87-a1e1bef565e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.255142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-760b391f-0e04-41b0-8f87-a1e1bef565e8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:32.256370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-760b391f-0e04-41b0-8f87-a1e1bef565e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:32.298360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7296e4-9340-4ac0-af08-3b96f057bc10 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.298670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7296e4-9340-4ac0-af08-3b96f057bc10 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.298912 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7296e4-9340-4ac0-af08-3b96f057bc10 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.300414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 13/25] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:32.310288 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43473-2714-42a0-bee5-e6daf3012ad4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.310571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43473-2714-42a0-bee5-e6daf3012ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.310841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43473-2714-42a0-bee5-e6daf3012ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.324337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.333018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 13/26] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:32.342882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f71df-ff62-4cc0-80c9-8b8d8c65fe5d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.343062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-49a43473-2714-42a0-bee5-e6daf3012ad4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:32.343560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f71df-ff62-4cc0-80c9-8b8d8c65fe5d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.344006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f71df-ff62-4cc0-80c9-8b8d8c65fe5d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.344281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49a43473-2714-42a0-bee5-e6daf3012ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:32.348434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 14/27] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:32.353701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c954c8-d02d-4b80-be65-4edce745e04b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.354406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c954c8-d02d-4b80-be65-4edce745e04b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.354860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c954c8-d02d-4b80-be65-4edce745e04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.383029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f0c954c8-d02d-4b80-be65-4edce745e04b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:32.385607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f0c954c8-d02d-4b80-be65-4edce745e04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:32.459039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 14/28] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:32.486872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 15/29] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:32.616426 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a43473-2714-42a0-bee5-e6daf3012ad4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.629407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.738640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a43473-2714-42a0-bee5-e6daf3012ad4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.786125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008b54b-67a7-4235-8597-6cd7cc8359eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.786733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b54b-67a7-4235-8597-6cd7cc8359eb None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.787006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b54b-67a7-4235-8597-6cd7cc8359eb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.787274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b54b-67a7-4235-8597-6cd7cc8359eb None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.787946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008b54b-67a7-4235-8597-6cd7cc8359eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.789399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008b54b-67a7-4235-8597-6cd7cc8359eb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.792519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c954c8-d02d-4b80-be65-4edce745e04b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.976573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c954c8-d02d-4b80-be65-4edce745e04b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.977361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 15/30] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Nov 2 07:14:32 2022] POST /identity/v3/roles => generated 219 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:32.978829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008b54b-67a7-4235-8597-6cd7cc8359eb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.008945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.015892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe404fef-27a7-4db8-a537-d34fa93ec289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.016535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe404fef-27a7-4db8-a537-d34fa93ec289 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.017036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe404fef-27a7-4db8-a537-d34fa93ec289 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.017440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe404fef-27a7-4db8-a537-d34fa93ec289 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.029867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe404fef-27a7-4db8-a537-d34fa93ec289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-fe404fef-27a7-4db8-a537-d34fa93ec289 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:33.035439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:33.036301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:33.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:14:33.039607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 16/31] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:33.041223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe404fef-27a7-4db8-a537-d34fa93ec289 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.079988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.115639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008b54b-67a7-4235-8597-6cd7cc8359eb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.154429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e4f60-084e-48e3-a34c-a1dd8f78b9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.155422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4f60-084e-48e3-a34c-a1dd8f78b9c0 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.155891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4f60-084e-48e3-a34c-a1dd8f78b9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.156381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4f60-084e-48e3-a34c-a1dd8f78b9c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.157271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e4f60-084e-48e3-a34c-a1dd8f78b9c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.159034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e4f60-084e-48e3-a34c-a1dd8f78b9c0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.160805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe404fef-27a7-4db8-a537-d34fa93ec289 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.176586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 16/32] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Nov 2 07:14:32 2022] POST /identity/v3/roles => generated 225 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:33.177962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23774e2a-4052-45c7-a4cf-9670abdd271d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.178275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23774e2a-4052-45c7-a4cf-9670abdd271d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.178603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23774e2a-4052-45c7-a4cf-9670abdd271d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.179860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 17/33] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:33.180597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23774e2a-4052-45c7-a4cf-9670abdd271d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.201481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237f058-3c0b-4669-b027-bc3866744433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.202232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237f058-3c0b-4669-b027-bc3866744433 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.202521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237f058-3c0b-4669-b027-bc3866744433 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.202799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237f058-3c0b-4669-b027-bc3866744433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.215306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237f058-3c0b-4669-b027-bc3866744433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.216975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237f058-3c0b-4669-b027-bc3866744433 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9237f058-3c0b-4669-b027-bc3866744433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:33.234765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:33.235700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:33.236458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:33.236458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:33.236458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:14:33.236458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 17/34] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:33.237384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab7f52-69f8-422b-8fb5-52c812578482 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.237676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab7f52-69f8-422b-8fb5-52c812578482 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.237956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab7f52-69f8-422b-8fb5-52c812578482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.259084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-59ab7f52-69f8-422b-8fb5-52c812578482 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:33.260608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-59ab7f52-69f8-422b-8fb5-52c812578482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:33.301413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.303793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23774e2a-4052-45c7-a4cf-9670abdd271d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.335475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.337548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 18/35] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:33.338577 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab7f52-69f8-422b-8fb5-52c812578482 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.345871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.346820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.347274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.347715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.348566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:33.349073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.350821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.365708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 18/36] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:33.379217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.462560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab7f52-69f8-422b-8fb5-52c812578482 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.517359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ae140-a4ad-40e1-9563-3db82ce334ab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.562819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fe050-42fb-4f30-82b4-19dfde00472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.563838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe050-42fb-4f30-82b4-19dfde00472e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.564310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe050-42fb-4f30-82b4-19dfde00472e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.564764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe050-42fb-4f30-82b4-19dfde00472e None admin] PATH_INFO: `/v3/users/085301de455d4179ae28198d5b0bd385` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.566471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fe050-42fb-4f30-82b4-19dfde00472e None admin] RBAC: Authorizing `identity:get_user(user_id=085301de455d4179ae28198d5b0bd385)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.568164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fe050-42fb-4f30-82b4-19dfde00472e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.570028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 19/37] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/users/085301de455d4179ae28198d5b0bd385 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:33.579543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fe050-42fb-4f30-82b4-19dfde00472e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.617931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.618580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.618864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.619140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.620059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:33.620372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.622381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.644683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 19/38] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:33.650398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.654578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.693753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.694890 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fe050-42fb-4f30-82b4-19dfde00472e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.734687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823f5d7-9de6-4bf3-b7f8-ec4a5828bad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.736079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823f5d7-9de6-4bf3-b7f8-ec4a5828bad1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.736628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823f5d7-9de6-4bf3-b7f8-ec4a5828bad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.736918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823f5d7-9de6-4bf3-b7f8-ec4a5828bad1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.738477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823f5d7-9de6-4bf3-b7f8-ec4a5828bad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.739933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823f5d7-9de6-4bf3-b7f8-ec4a5828bad1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.746883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 20/39] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:33.746883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823f5d7-9de6-4bf3-b7f8-ec4a5828bad1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.777755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062152e-31a2-42e1-a221-a27f73959632 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.811497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8497c3-07cb-477f-ad55-a1574e7a2537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.812108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8497c3-07cb-477f-ad55-a1574e7a2537 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.812387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8497c3-07cb-477f-ad55-a1574e7a2537 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.812655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8497c3-07cb-477f-ad55-a1574e7a2537 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.813299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8497c3-07cb-477f-ad55-a1574e7a2537 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.814786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8497c3-07cb-477f-ad55-a1574e7a2537 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.847338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 20/40] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:33 2022] POST /identity/v3/projects => generated 301 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:14:33.848542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8497c3-07cb-477f-ad55-a1574e7a2537 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.872893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823f5d7-9de6-4bf3-b7f8-ec4a5828bad1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.904930 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.916934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324c4a-0142-4fdc-8954-ab1f3497455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.917913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324c4a-0142-4fdc-8954-ab1f3497455e None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.917913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324c4a-0142-4fdc-8954-ab1f3497455e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.918208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324c4a-0142-4fdc-8954-ab1f3497455e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.918906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c324c4a-0142-4fdc-8954-ab1f3497455e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.920470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c324c4a-0142-4fdc-8954-ab1f3497455e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.937902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 21/41] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Nov 2 07:14:33 2022] POST /identity/v3/roles => generated 223 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:33.939559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadfba7-46c3-42d0-b659-0344caa54b2c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.939855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadfba7-46c3-42d0-b659-0344caa54b2c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.940139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadfba7-46c3-42d0-b659-0344caa54b2c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.941381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 22/42] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:33.954182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a9df-c148-46e7-bf3f-480a823379db None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.954182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a9df-c148-46e7-bf3f-480a823379db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.954182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a9df-c148-46e7-bf3f-480a823379db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.969946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d9a1a9df-c148-46e7-bf3f-480a823379db None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:33.971776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d9a1a9df-c148-46e7-bf3f-480a823379db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:33.976616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.998882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8497c3-07cb-477f-ad55-a1574e7a2537 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.041003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.041996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.042310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.042598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.043450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.domain.id=default, user.id=085301de455d4179ae28198d5b0bd385) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:34.043780 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.045343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.053915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 21/43] 10.222.0.25 () {56 vars in 1322 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/role_assignments?user.id=085301de455d4179ae28198d5b0bd385&scope.domain.id=default&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 530 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:34.071570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 23/44] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:34.340795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.457698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87720480-f186-4bf4-9cdc-d04f938c6e34 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.493640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fce0ac-3a9c-4c56-9daa-ee851db45976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.494290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fce0ac-3a9c-4c56-9daa-ee851db45976 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.494580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fce0ac-3a9c-4c56-9daa-ee851db45976 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.494889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fce0ac-3a9c-4c56-9daa-ee851db45976 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.496075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fce0ac-3a9c-4c56-9daa-ee851db45976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.497546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fce0ac-3a9c-4c56-9daa-ee851db45976 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.499130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 22/45] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:34.505618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1a9df-c148-46e7-bf3f-480a823379db None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.613130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1a9df-c148-46e7-bf3f-480a823379db None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.630280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:34.656017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2910c-04ab-4c91-b009-b56cf61c6345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.656622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2910c-04ab-4c91-b009-b56cf61c6345 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.656901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2910c-04ab-4c91-b009-b56cf61c6345 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.657173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2910c-04ab-4c91-b009-b56cf61c6345 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.657823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2910c-04ab-4c91-b009-b56cf61c6345 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.659328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2910c-04ab-4c91-b009-b56cf61c6345 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.688093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:34 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:36.808672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067aa54-10ff-4317-9253-725201e416b6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067aa54-10ff-4317-9253-725201e416b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067aa54-10ff-4317-9253-725201e416b6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.811643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 23/47] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:36.817040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf0364-c454-40da-9819-0efa1cdd498b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.817446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf0364-c454-40da-9819-0efa1cdd498b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.817560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf0364-c454-40da-9819-0efa1cdd498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.839222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3cbf0364-c454-40da-9819-0efa1cdd498b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:36.840260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3cbf0364-c454-40da-9819-0efa1cdd498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:36.920100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 25/48] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:37.186932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067aa54-10ff-4317-9253-725201e416b6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.216684 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.312833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067aa54-10ff-4317-9253-725201e416b6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.353417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eee692-fb55-4961-90df-70ecb8cc2a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.354267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eee692-fb55-4961-90df-70ecb8cc2a19 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.354860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eee692-fb55-4961-90df-70ecb8cc2a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.355135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eee692-fb55-4961-90df-70ecb8cc2a19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.356306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eee692-fb55-4961-90df-70ecb8cc2a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.357784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eee692-fb55-4961-90df-70ecb8cc2a19 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.359370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 24/49] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:37.365820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf0364-c454-40da-9819-0efa1cdd498b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.478976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.482512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf0364-c454-40da-9819-0efa1cdd498b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.518657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952b5b4-0e2f-4244-ad2f-d1c4e2501a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.519253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b5b4-0e2f-4244-ad2f-d1c4e2501a6b None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.519527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b5b4-0e2f-4244-ad2f-d1c4e2501a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.519795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952b5b4-0e2f-4244-ad2f-d1c4e2501a6b None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.520434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952b5b4-0e2f-4244-ad2f-d1c4e2501a6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.521869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952b5b4-0e2f-4244-ad2f-d1c4e2501a6b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.549419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 26/50] 10.222.0.25 () {60 vars in 1140 bytes} [Wed Nov 2 07:14:37 2022] POST /identity/v3/projects => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:39.703980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd279e8c-1d12-49f7-91a2-6f9498ee6c4d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.704520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd279e8c-1d12-49f7-91a2-6f9498ee6c4d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.704520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd279e8c-1d12-49f7-91a2-6f9498ee6c4d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.706685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 25/51] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:39.714885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c4a9-a9d5-4e36-9256-78dfbfbc2d7f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.715328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c4a9-a9d5-4e36-9256-78dfbfbc2d7f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.715437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c4a9-a9d5-4e36-9256-78dfbfbc2d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.735020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b9d7c4a9-a9d5-4e36-9256-78dfbfbc2d7f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:39.736203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b9d7c4a9-a9d5-4e36-9256-78dfbfbc2d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:39.805185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 27/52] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:40.074369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd279e8c-1d12-49f7-91a2-6f9498ee6c4d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.174226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.191642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd279e8c-1d12-49f7-91a2-6f9498ee6c4d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.226826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf983b3-a636-4e27-8242-2cc3780c2be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.227444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf983b3-a636-4e27-8242-2cc3780c2be9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.227721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf983b3-a636-4e27-8242-2cc3780c2be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.227990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf983b3-a636-4e27-8242-2cc3780c2be9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.229395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf983b3-a636-4e27-8242-2cc3780c2be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.230879 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf983b3-a636-4e27-8242-2cc3780c2be9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.232423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:40.240484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7c4a9-a9d5-4e36-9256-78dfbfbc2d7f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.322066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.342867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7c4a9-a9d5-4e36-9256-78dfbfbc2d7f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.377737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5ed21-ade0-4194-a000-acf54e4a62f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.378361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5ed21-ade0-4194-a000-acf54e4a62f3 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.378641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5ed21-ade0-4194-a000-acf54e4a62f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.378913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5ed21-ade0-4194-a000-acf54e4a62f3 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.379568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5ed21-ade0-4194-a000-acf54e4a62f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.381029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5ed21-ade0-4194-a000-acf54e4a62f3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.414110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 28/54] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:40 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:14:43.529374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6610-2184-4355-88a9-e5089a8abac0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.530449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6610-2184-4355-88a9-e5089a8abac0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.530975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6610-2184-4355-88a9-e5089a8abac0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.532537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 27/55] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:43.538332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082ab7-a6d0-43a5-8fd2-16ece485765d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.539187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082ab7-a6d0-43a5-8fd2-16ece485765d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.539680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082ab7-a6d0-43a5-8fd2-16ece485765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.566629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-27082ab7-a6d0-43a5-8fd2-16ece485765d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:43.568169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-27082ab7-a6d0-43a5-8fd2-16ece485765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:43.741549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28793eac-e88b-4723-98ba-dfcb0c653c7f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.742264 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28793eac-e88b-4723-98ba-dfcb0c653c7f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.742780 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28793eac-e88b-4723-98ba-dfcb0c653c7f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.744328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 28/56] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:43.750185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36b7-f1e2-45b1-8137-71eea3f724f4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.750737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36b7-f1e2-45b1-8137-71eea3f724f4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.751200 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f36b7-f1e2-45b1-8137-71eea3f724f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.775991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fd0f36b7-f1e2-45b1-8137-71eea3f724f4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:43.780398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd0f36b7-f1e2-45b1-8137-71eea3f724f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:43.802148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 29/57] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:43.858221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 29/58] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:43.983339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94333da1-8432-4e67-89c2-023b45508d7f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.983668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94333da1-8432-4e67-89c2-023b45508d7f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.983895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94333da1-8432-4e67-89c2-023b45508d7f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.985497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 30/59] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:43.990330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae92ae-4edd-46bc-b94b-0d9780512a95 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.990822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae92ae-4edd-46bc-b94b-0d9780512a95 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.991308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae92ae-4edd-46bc-b94b-0d9780512a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.014124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-50ae92ae-4edd-46bc-b94b-0d9780512a95 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:44.015600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-50ae92ae-4edd-46bc-b94b-0d9780512a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:44.111862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 30/60] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:44.167042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94333da1-8432-4e67-89c2-023b45508d7f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.168995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae92ae-4edd-46bc-b94b-0d9780512a95 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.189642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.243087 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.279189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94333da1-8432-4e67-89c2-023b45508d7f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.314206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe30ff-4cfb-46de-b489-13045216142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.315213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe30ff-4cfb-46de-b489-13045216142c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.315683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe30ff-4cfb-46de-b489-13045216142c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.316139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe30ff-4cfb-46de-b489-13045216142c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.327716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe30ff-4cfb-46de-b489-13045216142c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3fbe30ff-4cfb-46de-b489-13045216142c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:44.334348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:44.335200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:44.335960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:44.336508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae92ae-4edd-46bc-b94b-0d9780512a95 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.338647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 31/61] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:44.340135 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8b9f-cd16-4efb-8986-ea43120e6002 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.340627 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8b9f-cd16-4efb-8986-ea43120e6002 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.341082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8b9f-cd16-4efb-8986-ea43120e6002 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.342538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 32/62] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:44.344471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e8b9f-cd16-4efb-8986-ea43120e6002 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.380562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ae9c5-5aaa-4857-8ce9-7c567d0a3272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.381203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae9c5-5aaa-4857-8ce9-7c567d0a3272 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.381535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae9c5-5aaa-4857-8ce9-7c567d0a3272 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.381824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae9c5-5aaa-4857-8ce9-7c567d0a3272 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.393617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ae9c5-5aaa-4857-8ce9-7c567d0a3272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-803ae9c5-5aaa-4857-8ce9-7c567d0a3272 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:44.398918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:44.399740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 31/63] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:44.405518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb917a-f1b6-456c-88ac-d4701188f341 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.406222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb917a-f1b6-456c-88ac-d4701188f341 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.406222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb917a-f1b6-456c-88ac-d4701188f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.425965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a2cb917a-f1b6-456c-88ac-d4701188f341 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:44.427793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a2cb917a-f1b6-456c-88ac-d4701188f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:44.471800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e8b9f-cd16-4efb-8986-ea43120e6002 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.499477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.502043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 32/64] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:44.502853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb917a-f1b6-456c-88ac-d4701188f341 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.505334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.514735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.516043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.516889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.517616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.518551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:44.519273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.521637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.533602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 33/65] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:44.534824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.611056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb917a-f1b6-456c-88ac-d4701188f341 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.647203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.648188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.648731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.649314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.650255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:44.650814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.652522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.661863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 33/66] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:44.666408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b77e42-fd6c-413c-bc58-66cacac69ee2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.667804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.703535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44774fcd-dad5-4a4f-96cf-45cd11d627cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.704145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44774fcd-dad5-4a4f-96cf-45cd11d627cd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.704419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44774fcd-dad5-4a4f-96cf-45cd11d627cd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.704689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44774fcd-dad5-4a4f-96cf-45cd11d627cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.717065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44774fcd-dad5-4a4f-96cf-45cd11d627cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-44774fcd-dad5-4a4f-96cf-45cd11d627cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:44.721871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:44.722846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:44.723586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 34/67] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:44.724211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44774fcd-dad5-4a4f-96cf-45cd11d627cd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.740740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.780122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.780666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b7895-7130-4f93-8cd1-0067d5428b60 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.813365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69baf2e-908a-42c0-adfc-96e6db41335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.814000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69baf2e-908a-42c0-adfc-96e6db41335f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.814302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69baf2e-908a-42c0-adfc-96e6db41335f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.814580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69baf2e-908a-42c0-adfc-96e6db41335f None admin] PATH_INFO: `/v3/users/97212451ad45468b86698f04b5189e88` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.829344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69baf2e-908a-42c0-adfc-96e6db41335f None admin] RBAC: Authorizing `identity:get_user(user_id=97212451ad45468b86698f04b5189e88)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.830529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44774fcd-dad5-4a4f-96cf-45cd11d627cd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.836466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69baf2e-908a-42c0-adfc-96e6db41335f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.839273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 34/68] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/users/97212451ad45468b86698f04b5189e88 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:44.840445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eaf8a-7d34-4f9e-9efb-e54d2e3147b3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.840730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eaf8a-7d34-4f9e-9efb-e54d2e3147b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.841008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eaf8a-7d34-4f9e-9efb-e54d2e3147b3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.842237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 35/69] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:44.842932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3eaf8a-7d34-4f9e-9efb-e54d2e3147b3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.872931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85018142-4ed1-4e24-9f69-0c723e113fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.873564 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85018142-4ed1-4e24-9f69-0c723e113fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.873860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85018142-4ed1-4e24-9f69-0c723e113fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.874156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85018142-4ed1-4e24-9f69-0c723e113fc0 None admin] PATH_INFO: `/v3/users/085301de455d4179ae28198d5b0bd385` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.875326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85018142-4ed1-4e24-9f69-0c723e113fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=085301de455d4179ae28198d5b0bd385)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.876813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85018142-4ed1-4e24-9f69-0c723e113fc0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.882165 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 35/70] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/users/085301de455d4179ae28198d5b0bd385 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:44.882165 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85018142-4ed1-4e24-9f69-0c723e113fc0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.955828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3eaf8a-7d34-4f9e-9efb-e54d2e3147b3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.979736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.992217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.992578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85018142-4ed1-4e24-9f69-0c723e113fc0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.992825 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.993104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.993377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.994124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:44.994676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.996413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.004455 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 36/71] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.005072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.009785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.028370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52b3d9-582e-4b9b-9e54-6473d2f8a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.029044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52b3d9-582e-4b9b-9e54-6473d2f8a258 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.029325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52b3d9-582e-4b9b-9e54-6473d2f8a258 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.029598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52b3d9-582e-4b9b-9e54-6473d2f8a258 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.039875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52b3d9-582e-4b9b-9e54-6473d2f8a258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7c52b3d9-582e-4b9b-9e54-6473d2f8a258 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:45.045348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:45.046222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:45.046971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 36/72] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:45.047679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b138-a738-41cf-9145-1802a14dc310 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.048250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b138-a738-41cf-9145-1802a14dc310 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.048250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b138-a738-41cf-9145-1802a14dc310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.065364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ab06b138-a738-41cf-9145-1802a14dc310 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:45.066577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab06b138-a738-41cf-9145-1802a14dc310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:45.106191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff90db-01a7-4b18-8660-114cdc2c48ab None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.131958 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 37/73] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:45.132795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06b138-a738-41cf-9145-1802a14dc310 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.146172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88277a21-14e5-4188-acd1-3249da470fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.146815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88277a21-14e5-4188-acd1-3249da470fec None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.147099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88277a21-14e5-4188-acd1-3249da470fec None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.147377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88277a21-14e5-4188-acd1-3249da470fec None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.148515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88277a21-14e5-4188-acd1-3249da470fec None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.149997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88277a21-14e5-4188-acd1-3249da470fec None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.152285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 37/74] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.153054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88277a21-14e5-4188-acd1-3249da470fec None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.213592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.244371 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06b138-a738-41cf-9145-1802a14dc310 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.253752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.265601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88277a21-14e5-4188-acd1-3249da470fec None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.281426 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee0da-f497-405d-82f3-7bbf9694c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.282056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee0da-f497-405d-82f3-7bbf9694c352 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.282361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee0da-f497-405d-82f3-7bbf9694c352 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.282636 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee0da-f497-405d-82f3-7bbf9694c352 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.283775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ee0da-f497-405d-82f3-7bbf9694c352 None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.285222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ee0da-f497-405d-82f3-7bbf9694c352 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.286904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 38/75] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.291911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee0da-f497-405d-82f3-7bbf9694c352 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.306657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d2af-df11-4273-931f-7da067a81a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.307399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d2af-df11-4273-931f-7da067a81a8c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.307705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d2af-df11-4273-931f-7da067a81a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.307982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d2af-df11-4273-931f-7da067a81a8c None admin] PATH_INFO: `/v3/users/97212451ad45468b86698f04b5189e88` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.309235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d2af-df11-4273-931f-7da067a81a8c None admin] RBAC: Authorizing `identity:get_user(user_id=97212451ad45468b86698f04b5189e88)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.310841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d2af-df11-4273-931f-7da067a81a8c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.312563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 38/76] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/users/97212451ad45468b86698f04b5189e88 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.317723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d2af-df11-4273-931f-7da067a81a8c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.390431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee0da-f497-405d-82f3-7bbf9694c352 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.422390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d2af-df11-4273-931f-7da067a81a8c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.425462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.426077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.426385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.426663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.427327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:45.427632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.429029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.437389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 39/77] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.438188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.445936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.460427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665022-46ed-41bc-85d5-df0491f98aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.461086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665022-46ed-41bc-85d5-df0491f98aa9 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.461366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665022-46ed-41bc-85d5-df0491f98aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.461641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665022-46ed-41bc-85d5-df0491f98aa9 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0/users/97212451ad45468b86698f04b5189e88/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.463981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a665022-46ed-41bc-85d5-df0491f98aa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=61e389290239460b8b1a7846922850f0, user_id=97212451ad45468b86698f04b5189e88, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.465438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a665022-46ed-41bc-85d5-df0491f98aa9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.478574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 39/78] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:45 2022] PUT /identity/v3/projects/61e389290239460b8b1a7846922850f0/users/97212451ad45468b86698f04b5189e88/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:45.479660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665022-46ed-41bc-85d5-df0491f98aa9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.484687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.553395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83109ee-6121-44ec-89c0-07fdf5fc6e95 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.591694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502e71e-a7e5-4214-b946-9f57a49445f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.592384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e71e-a7e5-4214-b946-9f57a49445f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.592677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e71e-a7e5-4214-b946-9f57a49445f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.592956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e71e-a7e5-4214-b946-9f57a49445f3 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0/users/085301de455d4179ae28198d5b0bd385/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.594992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502e71e-a7e5-4214-b946-9f57a49445f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=61e389290239460b8b1a7846922850f0, user_id=085301de455d4179ae28198d5b0bd385, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.595294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665022-46ed-41bc-85d5-df0491f98aa9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.596479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502e71e-a7e5-4214-b946-9f57a49445f3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.618203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 40/79] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:45 2022] PUT /identity/v3/projects/61e389290239460b8b1a7846922850f0/users/085301de455d4179ae28198d5b0bd385/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:45.618203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502e71e-a7e5-4214-b946-9f57a49445f3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.655199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278a3fc-6705-4651-9911-ed40dfab95ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.656163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278a3fc-6705-4651-9911-ed40dfab95ba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.657458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278a3fc-6705-4651-9911-ed40dfab95ba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.657891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278a3fc-6705-4651-9911-ed40dfab95ba None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.660161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278a3fc-6705-4651-9911-ed40dfab95ba None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.661642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278a3fc-6705-4651-9911-ed40dfab95ba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.663946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 40/80] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.665014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278a3fc-6705-4651-9911-ed40dfab95ba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.731433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.736752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.742219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502e71e-a7e5-4214-b946-9f57a49445f3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.791548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f9274-3973-42ec-b75e-fbdfb62b4030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.792072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f9274-3973-42ec-b75e-fbdfb62b4030 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.792367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f9274-3973-42ec-b75e-fbdfb62b4030 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.792648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f9274-3973-42ec-b75e-fbdfb62b4030 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.793859 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f9274-3973-42ec-b75e-fbdfb62b4030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.795400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f9274-3973-42ec-b75e-fbdfb62b4030 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.795682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278a3fc-6705-4651-9911-ed40dfab95ba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.797038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 41/81] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.798295 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f9274-3973-42ec-b75e-fbdfb62b4030 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.835764 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4243b-4210-46ae-900d-678f2bb9e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.836721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4243b-4210-46ae-900d-678f2bb9e464 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.837472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4243b-4210-46ae-900d-678f2bb9e464 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.837751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4243b-4210-46ae-900d-678f2bb9e464 None admin] PATH_INFO: `/v3/users/97212451ad45468b86698f04b5189e88` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.841885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4243b-4210-46ae-900d-678f2bb9e464 None admin] RBAC: Authorizing `identity:get_user(user_id=97212451ad45468b86698f04b5189e88)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.843411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4243b-4210-46ae-900d-678f2bb9e464 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.845314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 41/82] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/users/97212451ad45468b86698f04b5189e88 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.846553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4243b-4210-46ae-900d-678f2bb9e464 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.933610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f9274-3973-42ec-b75e-fbdfb62b4030 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.949798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4243b-4210-46ae-900d-678f2bb9e464 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.967026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.972299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506939d-a1ab-4bac-9d5a-6e20a5bda75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.973267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506939d-a1ab-4bac-9d5a-6e20a5bda75f None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.973744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506939d-a1ab-4bac-9d5a-6e20a5bda75f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.974272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506939d-a1ab-4bac-9d5a-6e20a5bda75f None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0/users/97212451ad45468b86698f04b5189e88/roles/5a2ee20f691f47539f8b1850459be9ca` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.977533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506939d-a1ab-4bac-9d5a-6e20a5bda75f None admin] RBAC: Authorizing `identity:create_grant(project_id=61e389290239460b8b1a7846922850f0, user_id=97212451ad45468b86698f04b5189e88, role_id=5a2ee20f691f47539f8b1850459be9ca)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.979603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506939d-a1ab-4bac-9d5a-6e20a5bda75f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.985206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33660-9189-4295-a4a3-81c982485c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.985757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33660-9189-4295-a4a3-81c982485c70 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.986066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33660-9189-4295-a4a3-81c982485c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.986353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33660-9189-4295-a4a3-81c982485c70 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.986983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c33660-9189-4295-a4a3-81c982485c70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.988551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c33660-9189-4295-a4a3-81c982485c70 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.996722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 42/83] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:45 2022] PUT /identity/v3/projects/61e389290239460b8b1a7846922850f0/users/97212451ad45468b86698f04b5189e88/roles/5a2ee20f691f47539f8b1850459be9ca => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:45.998146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506939d-a1ab-4bac-9d5a-6e20a5bda75f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:46.028958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 42/84] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:45 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:46.033227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:46.122326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506939d-a1ab-4bac-9d5a-6e20a5bda75f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:46.163160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b9b8d-8cd2-423d-a747-ffa032ca8964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:46.164308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b9b8d-8cd2-423d-a747-ffa032ca8964 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:46.164783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b9b8d-8cd2-423d-a747-ffa032ca8964 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:46.165271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b9b8d-8cd2-423d-a747-ffa032ca8964 None admin] PATH_INFO: `/v3/projects/4946899193ea460893ddbc695e5625ff` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:46.166767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b9b8d-8cd2-423d-a747-ffa032ca8964 None admin] RBAC: Authorizing `identity:get_project(project_id=4946899193ea460893ddbc695e5625ff)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:46.168478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b9b8d-8cd2-423d-a747-ffa032ca8964 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:46.170672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 43/85] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/projects/4946899193ea460893ddbc695e5625ff => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:46.177198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33660-9189-4295-a4a3-81c982485c70 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:46.305704 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33660-9189-4295-a4a3-81c982485c70 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:46.342496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4418d2f-348e-41a6-978a-2feea2c6bc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:46.343175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418d2f-348e-41a6-978a-2feea2c6bc80 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:46.343459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418d2f-348e-41a6-978a-2feea2c6bc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:46.343730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418d2f-348e-41a6-978a-2feea2c6bc80 None admin] PATH_INFO: `/v3/projects/4946899193ea460893ddbc695e5625ff/users/97212451ad45468b86698f04b5189e88/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:46.345711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4418d2f-348e-41a6-978a-2feea2c6bc80 None admin] RBAC: Authorizing `identity:create_grant(project_id=4946899193ea460893ddbc695e5625ff, user_id=97212451ad45468b86698f04b5189e88, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:46.347198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4418d2f-348e-41a6-978a-2feea2c6bc80 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:46.364151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 43/86] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:46 2022] PUT /identity/v3/projects/4946899193ea460893ddbc695e5625ff/users/97212451ad45468b86698f04b5189e88/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:47.811519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc47f9-c928-44b1-8e03-9e16218d347c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:47.812040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc47f9-c928-44b1-8e03-9e16218d347c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:47.812114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc47f9-c928-44b1-8e03-9e16218d347c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:47.813568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 44/87] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:47.818736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a33e81-872b-4bcd-92d1-8fb3e17b15a3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:47.819202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a33e81-872b-4bcd-92d1-8fb3e17b15a3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:47.819277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a33e81-872b-4bcd-92d1-8fb3e17b15a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:47.847146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-26a33e81-872b-4bcd-92d1-8fb3e17b15a3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:47.848358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-26a33e81-872b-4bcd-92d1-8fb3e17b15a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:47.879342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:47.945063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 44/88] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:48.352053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e815-be99-4cc5-8d4b-bbe64c22ec71 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.352421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e815-be99-4cc5-8d4b-bbe64c22ec71 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.352617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e815-be99-4cc5-8d4b-bbe64c22ec71 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.354193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 45/89] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:48.359710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a8b6-0871-4cb7-a79b-9c0b135fd6d8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.360005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a8b6-0871-4cb7-a79b-9c0b135fd6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.360286 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a8b6-0871-4cb7-a79b-9c0b135fd6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.382411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7370a8b6-0871-4cb7-a79b-9c0b135fd6d8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:48.383608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7370a8b6-0871-4cb7-a79b-9c0b135fd6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:48.417875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3e815-be99-4cc5-8d4b-bbe64c22ec71 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.500145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 45/90] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:48.555412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:48.570784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3e815-be99-4cc5-8d4b-bbe64c22ec71 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:48.614444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e309c70-7589-4c98-af2b-1ca1ef6191ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:48.615533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e309c70-7589-4c98-af2b-1ca1ef6191ee None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.616002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e309c70-7589-4c98-af2b-1ca1ef6191ee None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.616461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e309c70-7589-4c98-af2b-1ca1ef6191ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.630035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e309c70-7589-4c98-af2b-1ca1ef6191ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9e309c70-7589-4c98-af2b-1ca1ef6191ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:48.635362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:48.636178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:48.636927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:48.639123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 46/91] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:48.645208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370a8b6-0871-4cb7-a79b-9c0b135fd6d8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.753861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:48.764005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370a8b6-0871-4cb7-a79b-9c0b135fd6d8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:48.808777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:48.809746 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.810238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.810699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.811614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:48.812172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:48.813732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:48.822448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 46/92] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:48.831802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e309c70-7589-4c98-af2b-1ca1ef6191ee None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.975920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e309c70-7589-4c98-af2b-1ca1ef6191ee None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:48.981266 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.043999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2faa9-40b7-4c37-b7fd-445ecf7c3ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.044663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2faa9-40b7-4c37-b7fd-445ecf7c3ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.044948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2faa9-40b7-4c37-b7fd-445ecf7c3ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.045226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2faa9-40b7-4c37-b7fd-445ecf7c3ac6 None admin] PATH_INFO: `/v3/users/97212451ad45468b86698f04b5189e88` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.050920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2faa9-40b7-4c37-b7fd-445ecf7c3ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=97212451ad45468b86698f04b5189e88)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.052424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2faa9-40b7-4c37-b7fd-445ecf7c3ac6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.054116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 47/93] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/users/97212451ad45468b86698f04b5189e88 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:49.059936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2faa9-40b7-4c37-b7fd-445ecf7c3ac6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.108759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.162855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b270529-ce92-475a-9dc1-6685023944ac None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.202847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19d24e-431e-4f80-81bb-e8ed0c4d2387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.204265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d24e-431e-4f80-81bb-e8ed0c4d2387 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.204772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d24e-431e-4f80-81bb-e8ed0c4d2387 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.205264 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d24e-431e-4f80-81bb-e8ed0c4d2387 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.217500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d24e-431e-4f80-81bb-e8ed0c4d2387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7e19d24e-431e-4f80-81bb-e8ed0c4d2387 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:49.224349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:49.225191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:49.225914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:14:49.228717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 47/94] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:49.230255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216c63f-595c-42e1-b765-37181923e3e6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.231206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216c63f-595c-42e1-b765-37181923e3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.231669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216c63f-595c-42e1-b765-37181923e3e6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.233757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 48/95] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:49.234712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216c63f-595c-42e1-b765-37181923e3e6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.266471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.277305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2faa9-40b7-4c37-b7fd-445ecf7c3ac6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.344793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be540da0-1f71-4b72-9e40-ff3ad734280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.346000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be540da0-1f71-4b72-9e40-ff3ad734280b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.346579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be540da0-1f71-4b72-9e40-ff3ad734280b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.347180 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be540da0-1f71-4b72-9e40-ff3ad734280b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.376497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216c63f-595c-42e1-b765-37181923e3e6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.444564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d29d4d-82eb-4aab-9855-c9580c5550b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.445529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d29d4d-82eb-4aab-9855-c9580c5550b2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.446658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d29d4d-82eb-4aab-9855-c9580c5550b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.447284 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d29d4d-82eb-4aab-9855-c9580c5550b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.449539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d29d4d-82eb-4aab-9855-c9580c5550b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:49.450047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d29d4d-82eb-4aab-9855-c9580c5550b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.452396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d29d4d-82eb-4aab-9855-c9580c5550b2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.464300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be540da0-1f71-4b72-9e40-ff3ad734280b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.466201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be540da0-1f71-4b72-9e40-ff3ad734280b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.472428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 49/96] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:49.474914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dece2f7-1266-4a23-a12a-9be48206ecd3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.476761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dece2f7-1266-4a23-a12a-9be48206ecd3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.477236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dece2f7-1266-4a23-a12a-9be48206ecd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.503378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6dece2f7-1266-4a23-a12a-9be48206ecd3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:49.504936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6dece2f7-1266-4a23-a12a-9be48206ecd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:49.526515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.575906 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 50/97] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:49.577152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dece2f7-1266-4a23-a12a-9be48206ecd3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.585653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.625896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 48/98] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:49.631476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215c74b-dfb2-44b6-a4c1-b5cbd0dd8724 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.631769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215c74b-dfb2-44b6-a4c1-b5cbd0dd8724 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.632047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215c74b-dfb2-44b6-a4c1-b5cbd0dd8724 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.641385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 49/99] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:49 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:49.642071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215c74b-dfb2-44b6-a4c1-b5cbd0dd8724 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.764374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dece2f7-1266-4a23-a12a-9be48206ecd3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.820131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215c74b-dfb2-44b6-a4c1-b5cbd0dd8724 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.840432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7462aa1-89c3-4c8c-ae4f-7b910e95be43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.841079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7462aa1-89c3-4c8c-ae4f-7b910e95be43 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.841352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7462aa1-89c3-4c8c-ae4f-7b910e95be43 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.841628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7462aa1-89c3-4c8c-ae4f-7b910e95be43 None admin] PATH_INFO: `/v3/users/085301de455d4179ae28198d5b0bd385` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.844405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7462aa1-89c3-4c8c-ae4f-7b910e95be43 None admin] RBAC: Authorizing `identity:get_user(user_id=085301de455d4179ae28198d5b0bd385)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.845877 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7462aa1-89c3-4c8c-ae4f-7b910e95be43 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.847522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 51/100] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/users/085301de455d4179ae28198d5b0bd385 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:49.856490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb15b5-9cd2-47b7-ba9c-c39e984dd089 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.856776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb15b5-9cd2-47b7-ba9c-c39e984dd089 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.857059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb15b5-9cd2-47b7-ba9c-c39e984dd089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.872435 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.894505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4807a-c0c3-4fad-b295-1eca43543f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.895204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4807a-c0c3-4fad-b295-1eca43543f30 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.898360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4807a-c0c3-4fad-b295-1eca43543f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.898360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4807a-c0c3-4fad-b295-1eca43543f30 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.898360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4807a-c0c3-4fad-b295-1eca43543f30 None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.898618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4807a-c0c3-4fad-b295-1eca43543f30 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.900269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 50/101] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:49.901471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4807a-c0c3-4fad-b295-1eca43543f30 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.903354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f3cb15b5-9cd2-47b7-ba9c-c39e984dd089 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:49.904773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f3cb15b5-9cd2-47b7-ba9c-c39e984dd089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:49.978923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 52/102] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:49.981515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c245a9-8ce4-4338-908f-d32f31f6dbb8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.981807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c245a9-8ce4-4338-908f-d32f31f6dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.982109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c245a9-8ce4-4338-908f-d32f31f6dbb8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.983465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 53/103] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:49.987610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c245a9-8ce4-4338-908f-d32f31f6dbb8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.988127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.016635 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4807a-c0c3-4fad-b295-1eca43543f30 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.056270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89028f-5b40-489c-998e-8d8187c575df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.057011 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89028f-5b40-489c-998e-8d8187c575df None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.057299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89028f-5b40-489c-998e-8d8187c575df None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.057584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89028f-5b40-489c-998e-8d8187c575df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.089391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c245a9-8ce4-4338-908f-d32f31f6dbb8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.119237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.125524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf0f39-6a15-41d1-aa84-c08a8f1b7a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.126156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf0f39-6a15-41d1-aa84-c08a8f1b7a05 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.126440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf0f39-6a15-41d1-aa84-c08a8f1b7a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.126715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf0f39-6a15-41d1-aa84-c08a8f1b7a05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.127446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf0f39-6a15-41d1-aa84-c08a8f1b7a05 None admin] RBAC: Adding query filter params (role.id=2e66bd66105e4a1e9dcbda045dc2d057, scope.project.id=61e389290239460b8b1a7846922850f0, user.id=97212451ad45468b86698f04b5189e88) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.127757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf0f39-6a15-41d1-aa84-c08a8f1b7a05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.129221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf0f39-6a15-41d1-aa84-c08a8f1b7a05 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.130668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89028f-5b40-489c-998e-8d8187c575df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.132328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89028f-5b40-489c-998e-8d8187c575df None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.135459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 54/104] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/role_assignments?user.id=97212451ad45468b86698f04b5189e88&scope.project.id=61e389290239460b8b1a7846922850f0&role.id=2e66bd66105e4a1e9dcbda045dc2d057 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.136512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8451a6e-9f60-4809-aa68-428546174704 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.136801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8451a6e-9f60-4809-aa68-428546174704 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.137082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8451a6e-9f60-4809-aa68-428546174704 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.138494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 55/105] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:50.139125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8451a6e-9f60-4809-aa68-428546174704 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.237375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.266502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 51/106] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:50.267843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44ddd9-1c41-4ac5-be51-8f6c541a1b65 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.268082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44ddd9-1c41-4ac5-be51-8f6c541a1b65 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.268362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44ddd9-1c41-4ac5-be51-8f6c541a1b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.287588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2d44ddd9-1c41-4ac5-be51-8f6c541a1b65 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:50.288824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d44ddd9-1c41-4ac5-be51-8f6c541a1b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:50.289049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8451a6e-9f60-4809-aa68-428546174704 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.329812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8f836-c58e-478f-b9db-dc650f7f7c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.330012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8f836-c58e-478f-b9db-dc650f7f7c19 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.330237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8f836-c58e-478f-b9db-dc650f7f7c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.330495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8f836-c58e-478f-b9db-dc650f7f7c19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.341869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8f836-c58e-478f-b9db-dc650f7f7c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4db8f836-c58e-478f-b9db-dc650f7f7c19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:50.347190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:50.348056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:14:50.348937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 56/107] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:50.349686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178b9d-7ca7-48e6-840f-67d96185d857 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.349686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178b9d-7ca7-48e6-840f-67d96185d857 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.349945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178b9d-7ca7-48e6-840f-67d96185d857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.368967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3b178b9d-7ca7-48e6-840f-67d96185d857 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:50.370314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b178b9d-7ca7-48e6-840f-67d96185d857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:50.371539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 52/108] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:50.372403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44ddd9-1c41-4ac5-be51-8f6c541a1b65 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.393224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.448841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 57/109] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:50.449983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b178b9d-7ca7-48e6-840f-67d96185d857 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.484983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44ddd9-1c41-4ac5-be51-8f6c541a1b65 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.518789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.528731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208e1d1-85d9-4e30-a3fe-68352b2a6f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.529380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208e1d1-85d9-4e30-a3fe-68352b2a6f2d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.529663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208e1d1-85d9-4e30-a3fe-68352b2a6f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.529935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208e1d1-85d9-4e30-a3fe-68352b2a6f2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.542043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208e1d1-85d9-4e30-a3fe-68352b2a6f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8208e1d1-85d9-4e30-a3fe-68352b2a6f2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:50.547836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:50.548685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:50.549442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:50.551125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 53/110] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:50.556023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208e1d1-85d9-4e30-a3fe-68352b2a6f2d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.577072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b178b9d-7ca7-48e6-840f-67d96185d857 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.621870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91b90-2583-4a97-8b43-ff0259d63e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.623576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91b90-2583-4a97-8b43-ff0259d63e49 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.624059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91b90-2583-4a97-8b43-ff0259d63e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.624772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91b90-2583-4a97-8b43-ff0259d63e49 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.626794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b91b90-2583-4a97-8b43-ff0259d63e49 None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.629445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b91b90-2583-4a97-8b43-ff0259d63e49 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.631323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 58/111] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.632307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91b90-2583-4a97-8b43-ff0259d63e49 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.653925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.674664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208e1d1-85d9-4e30-a3fe-68352b2a6f2d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.712978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.713593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.713876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.714199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.714852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.715155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.716553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.725124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 54/112] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.725742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.738871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91b90-2583-4a97-8b43-ff0259d63e49 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.764713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.774482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.774990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.775269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.775683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.776366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.776676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.778057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.787589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 59/113] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.787589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.835699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54b67c-e244-4aa9-8e10-74550dc324e4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.876496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bc371-1c24-48ec-a734-8aa746a167d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.877132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc371-1c24-48ec-a734-8aa746a167d6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.877386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc371-1c24-48ec-a734-8aa746a167d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.877656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc371-1c24-48ec-a734-8aa746a167d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.878763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bc371-1c24-48ec-a734-8aa746a167d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.880215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bc371-1c24-48ec-a734-8aa746a167d6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.881754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 55/114] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.882994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bc371-1c24-48ec-a734-8aa746a167d6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.889727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e029b7-e4d8-4486-8b1c-dd1e7ba04036 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.899555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.926443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.927058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.927340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.927613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.928337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=61e389290239460b8b1a7846922850f0, user.id=085301de455d4179ae28198d5b0bd385) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.928647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.930098 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.940351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 60/115] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/role_assignments?user.id=085301de455d4179ae28198d5b0bd385&scope.project.id=61e389290239460b8b1a7846922850f0&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.943046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.997542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bc371-1c24-48ec-a734-8aa746a167d6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.001530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.036852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbc7eb-ea11-4798-968e-cd36d7ea186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.037420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbc7eb-ea11-4798-968e-cd36d7ea186f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.037701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbc7eb-ea11-4798-968e-cd36d7ea186f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.037981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbc7eb-ea11-4798-968e-cd36d7ea186f None admin] PATH_INFO: `/v3/users/97212451ad45468b86698f04b5189e88` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.039209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbc7eb-ea11-4798-968e-cd36d7ea186f None admin] RBAC: Authorizing `identity:get_user(user_id=97212451ad45468b86698f04b5189e88)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.040679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbc7eb-ea11-4798-968e-cd36d7ea186f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.047391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 56/116] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/users/97212451ad45468b86698f04b5189e88 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.047391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbc7eb-ea11-4798-968e-cd36d7ea186f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.061961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc7e58-8dab-4283-a15e-dcb596b2de8f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.100590 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d146a5b-a427-45db-9ec5-eb1b2835cf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.101333 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d146a5b-a427-45db-9ec5-eb1b2835cf7a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.101675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d146a5b-a427-45db-9ec5-eb1b2835cf7a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.102016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d146a5b-a427-45db-9ec5-eb1b2835cf7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.113166 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d146a5b-a427-45db-9ec5-eb1b2835cf7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1d146a5b-a427-45db-9ec5-eb1b2835cf7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:51.120539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:51.121423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 61/117] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:51.122221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d146a5b-a427-45db-9ec5-eb1b2835cf7a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.144201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.174269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbc7eb-ea11-4798-968e-cd36d7ea186f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.209139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d64ab4-7dd7-456c-9241-9d15b0c6619a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.209781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d64ab4-7dd7-456c-9241-9d15b0c6619a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.210065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d64ab4-7dd7-456c-9241-9d15b0c6619a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.210532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d64ab4-7dd7-456c-9241-9d15b0c6619a None admin] PATH_INFO: `/v3/users/97212451ad45468b86698f04b5189e88` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.211651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d64ab4-7dd7-456c-9241-9d15b0c6619a None admin] RBAC: Authorizing `identity:get_user(user_id=97212451ad45468b86698f04b5189e88)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.213503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d64ab4-7dd7-456c-9241-9d15b0c6619a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.215203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 57/118] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/users/97212451ad45468b86698f04b5189e88 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.218454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d64ab4-7dd7-456c-9241-9d15b0c6619a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.230056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d146a5b-a427-45db-9ec5-eb1b2835cf7a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.242661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.264257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d4be5-a6dd-420b-9f24-3decece83aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.264857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d4be5-a6dd-420b-9f24-3decece83aa4 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.265135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d4be5-a6dd-420b-9f24-3decece83aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.265410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d4be5-a6dd-420b-9f24-3decece83aa4 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.266072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d4be5-a6dd-420b-9f24-3decece83aa4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.267564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d4be5-a6dd-420b-9f24-3decece83aa4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.296101 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 62/119] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:51 2022] POST /identity/v3/projects => generated 302 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:51.296850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d4be5-a6dd-420b-9f24-3decece83aa4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.333000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d64ab4-7dd7-456c-9241-9d15b0c6619a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.373705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a04c7c-f9b9-447c-a243-93685b8ed6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.374367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a04c7c-f9b9-447c-a243-93685b8ed6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.374658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a04c7c-f9b9-447c-a243-93685b8ed6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.374923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a04c7c-f9b9-447c-a243-93685b8ed6f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.394325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.410073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d4be5-a6dd-420b-9f24-3decece83aa4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.447594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.448444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.448530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.448853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.449599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:51.449968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.451428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.457091 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a04c7c-f9b9-447c-a243-93685b8ed6f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.458632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a04c7c-f9b9-447c-a243-93685b8ed6f3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.460818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 63/120] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.462176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.493770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.574767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428dd0cf-9bf2-43ff-b9dd-754b9e885293 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.580526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 58/121] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:51.580526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a04c7c-f9b9-447c-a243-93685b8ed6f3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.618516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58556ad5-001b-4ab6-8e1d-e92e2dfb3460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.619320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58556ad5-001b-4ab6-8e1d-e92e2dfb3460 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.619589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58556ad5-001b-4ab6-8e1d-e92e2dfb3460 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.619813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58556ad5-001b-4ab6-8e1d-e92e2dfb3460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.626788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.682498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a04c7c-f9b9-447c-a243-93685b8ed6f3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.692719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58556ad5-001b-4ab6-8e1d-e92e2dfb3460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.694322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58556ad5-001b-4ab6-8e1d-e92e2dfb3460 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.720459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd785afa-8b1d-404d-99c3-74e40d6324ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.721095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd785afa-8b1d-404d-99c3-74e40d6324ab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.721376 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd785afa-8b1d-404d-99c3-74e40d6324ab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.721648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd785afa-8b1d-404d-99c3-74e40d6324ab None admin] PATH_INFO: `/v3/users/8ded85b99b10406890d1e839bb8f875d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.722326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.738911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd785afa-8b1d-404d-99c3-74e40d6324ab None admin] RBAC: Authorizing `identity:get_user(user_id=8ded85b99b10406890d1e839bb8f875d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.740578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd785afa-8b1d-404d-99c3-74e40d6324ab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.742361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 59/122] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/users/8ded85b99b10406890d1e839bb8f875d => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.748168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd785afa-8b1d-404d-99c3-74e40d6324ab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.800486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 64/123] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:51.801442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58556ad5-001b-4ab6-8e1d-e92e2dfb3460 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.859726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd785afa-8b1d-404d-99c3-74e40d6324ab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.862118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.899645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611dfa37-d562-4669-b512-3425092656f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.900408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dfa37-d562-4669-b512-3425092656f4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.901135 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dfa37-d562-4669-b512-3425092656f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.901403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dfa37-d562-4669-b512-3425092656f4 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.902566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611dfa37-d562-4669-b512-3425092656f4 None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.903691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58556ad5-001b-4ab6-8e1d-e92e2dfb3460 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.904059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611dfa37-d562-4669-b512-3425092656f4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.910529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 60/124] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.910529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611dfa37-d562-4669-b512-3425092656f4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.937410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.946820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533efcef-249f-40f4-a1dc-64911fcd54ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.947553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533efcef-249f-40f4-a1dc-64911fcd54ce None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.947836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533efcef-249f-40f4-a1dc-64911fcd54ce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.948113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533efcef-249f-40f4-a1dc-64911fcd54ce None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.949244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533efcef-249f-40f4-a1dc-64911fcd54ce None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.950784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533efcef-249f-40f4-a1dc-64911fcd54ce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.956991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 65/125] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.956991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533efcef-249f-40f4-a1dc-64911fcd54ce None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.020933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611dfa37-d562-4669-b512-3425092656f4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.054690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533efcef-249f-40f4-a1dc-64911fcd54ce None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.057738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21cee5-b0ea-4178-88b7-4dc9d911ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.057881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21cee5-b0ea-4178-88b7-4dc9d911ae9a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.058175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21cee5-b0ea-4178-88b7-4dc9d911ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.058459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21cee5-b0ea-4178-88b7-4dc9d911ae9a None admin] PATH_INFO: `/v3/projects/4946899193ea460893ddbc695e5625ff` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.059615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21cee5-b0ea-4178-88b7-4dc9d911ae9a None admin] RBAC: Authorizing `identity:get_project(project_id=4946899193ea460893ddbc695e5625ff)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.061094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21cee5-b0ea-4178-88b7-4dc9d911ae9a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.062940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 61/126] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/projects/4946899193ea460893ddbc695e5625ff => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:52.068331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21cee5-b0ea-4178-88b7-4dc9d911ae9a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.091914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.092546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.092836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.093094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.093889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] RBAC: Adding query filter params (role.id=5a2ee20f691f47539f8b1850459be9ca, scope.project.id=61e389290239460b8b1a7846922850f0, user.id=97212451ad45468b86698f04b5189e88) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:52.094228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.095671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.101355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 66/127] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/role_assignments?user.id=97212451ad45468b86698f04b5189e88&scope.project.id=61e389290239460b8b1a7846922850f0&role.id=5a2ee20f691f47539f8b1850459be9ca => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:52.102161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.123955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.185722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.205317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21cee5-b0ea-4178-88b7-4dc9d911ae9a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.245833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff16a6e-44c9-46e0-91a6-ac383de1986a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.251666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42c312-22ec-4aa7-996b-6a56176779d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.252346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42c312-22ec-4aa7-996b-6a56176779d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.252632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42c312-22ec-4aa7-996b-6a56176779d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.252897 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42c312-22ec-4aa7-996b-6a56176779d5 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0/users/8ded85b99b10406890d1e839bb8f875d/roles/0e99461723d34ada9c430e01b0c9ae3d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.255249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42c312-22ec-4aa7-996b-6a56176779d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=61e389290239460b8b1a7846922850f0, user_id=8ded85b99b10406890d1e839bb8f875d, role_id=0e99461723d34ada9c430e01b0c9ae3d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.256676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42c312-22ec-4aa7-996b-6a56176779d5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.275310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 62/128] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:52 2022] PUT /identity/v3/projects/61e389290239460b8b1a7846922850f0/users/8ded85b99b10406890d1e839bb8f875d/roles/0e99461723d34ada9c430e01b0c9ae3d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:52.292810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fd8e9-a5ed-48da-acc0-5992ed1ed691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.293771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fd8e9-a5ed-48da-acc0-5992ed1ed691 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.294269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fd8e9-a5ed-48da-acc0-5992ed1ed691 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.294780 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fd8e9-a5ed-48da-acc0-5992ed1ed691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.295718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fd8e9-a5ed-48da-acc0-5992ed1ed691 None admin] RBAC: Adding query filter params (role.id=2e66bd66105e4a1e9dcbda045dc2d057, scope.project.id=4946899193ea460893ddbc695e5625ff, user.id=97212451ad45468b86698f04b5189e88) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:52.296177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fd8e9-a5ed-48da-acc0-5992ed1ed691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.297851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fd8e9-a5ed-48da-acc0-5992ed1ed691 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.305229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 67/129] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/role_assignments?user.id=97212451ad45468b86698f04b5189e88&scope.project.id=4946899193ea460893ddbc695e5625ff&role.id=2e66bd66105e4a1e9dcbda045dc2d057 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.490233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed777fb-e03e-4f12-9a23-48927e11ac99 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.490766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed777fb-e03e-4f12-9a23-48927e11ac99 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.490766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed777fb-e03e-4f12-9a23-48927e11ac99 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.492120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 63/130] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:53.497505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0fb2-c5dc-4d1f-af25-5206ccd0520d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.497943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0fb2-c5dc-4d1f-af25-5206ccd0520d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.498061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0fb2-c5dc-4d1f-af25-5206ccd0520d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.517413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f2da0fb2-c5dc-4d1f-af25-5206ccd0520d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:53.518648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f2da0fb2-c5dc-4d1f-af25-5206ccd0520d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:53.589954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 68/131] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:53.850608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed777fb-e03e-4f12-9a23-48927e11ac99 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.957543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.962395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed777fb-e03e-4f12-9a23-48927e11ac99 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.997320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a2fee-93db-4f9d-8b5c-570cbf4bc21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.997939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2fee-93db-4f9d-8b5c-570cbf4bc21c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.998246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2fee-93db-4f9d-8b5c-570cbf4bc21c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.998512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2fee-93db-4f9d-8b5c-570cbf4bc21c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.999642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a2fee-93db-4f9d-8b5c-570cbf4bc21c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:54.001098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a2fee-93db-4f9d-8b5c-570cbf4bc21c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:54.002705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 64/132] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:54.008448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da0fb2-c5dc-4d1f-af25-5206ccd0520d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:54.061411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:54.108980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da0fb2-c5dc-4d1f-af25-5206ccd0520d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:54.143631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2bb6f-cf00-495b-9330-b06d82a7eb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:54.144236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2bb6f-cf00-495b-9330-b06d82a7eb1b None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.144811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2bb6f-cf00-495b-9330-b06d82a7eb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.145089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2bb6f-cf00-495b-9330-b06d82a7eb1b None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.145737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2bb6f-cf00-495b-9330-b06d82a7eb1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:54.147210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2bb6f-cf00-495b-9330-b06d82a7eb1b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:54.169978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 69/133] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:54 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:54.410306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83a90-1d54-495e-866d-f943ce33e1b6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.410628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83a90-1d54-495e-866d-f943ce33e1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.410904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83a90-1d54-495e-866d-f943ce33e1b6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.412241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 65/134] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:54.417837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413cadc-4741-46c9-83cf-3ca96c77f21e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.418182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413cadc-4741-46c9-83cf-3ca96c77f21e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.418454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413cadc-4741-46c9-83cf-3ca96c77f21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.437905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c413cadc-4741-46c9-83cf-3ca96c77f21e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:54.439184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c413cadc-4741-46c9-83cf-3ca96c77f21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:54.512547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 70/135] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:54.778055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83a90-1d54-495e-866d-f943ce33e1b6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:54.880103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83a90-1d54-495e-866d-f943ce33e1b6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:54.918279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aba379-e403-48e6-9114-e94b4f15e321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:54.918933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aba379-e403-48e6-9114-e94b4f15e321 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.919203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aba379-e403-48e6-9114-e94b4f15e321 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.919474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aba379-e403-48e6-9114-e94b4f15e321 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.934243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aba379-e403-48e6-9114-e94b4f15e321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b4aba379-e403-48e6-9114-e94b4f15e321 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:54.939125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:54.939945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:54.940690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 66/136] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:54.946622 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413cadc-4741-46c9-83cf-3ca96c77f21e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:54.994507 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:55.058662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413cadc-4741-46c9-83cf-3ca96c77f21e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:55.095915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:55.096924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.097460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.097917 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.098821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:55.099315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:55.100963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:55.114229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 71/137] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:55.116708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aba379-e403-48e6-9114-e94b4f15e321 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:55.147279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:55.230323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aba379-e403-48e6-9114-e94b4f15e321 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:55.264156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3995d-a671-4422-aa17-3d322b9a5c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:55.264786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3995d-a671-4422-aa17-3d322b9a5c1b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.265062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3995d-a671-4422-aa17-3d322b9a5c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.265333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3995d-a671-4422-aa17-3d322b9a5c1b None admin] PATH_INFO: `/v3/users/8ded85b99b10406890d1e839bb8f875d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.266480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3995d-a671-4422-aa17-3d322b9a5c1b None admin] RBAC: Authorizing `identity:get_user(user_id=8ded85b99b10406890d1e839bb8f875d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:55.267928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3995d-a671-4422-aa17-3d322b9a5c1b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:55.269493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 67/138] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:55 2022] GET /identity/v3/users/8ded85b99b10406890d1e839bb8f875d => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:55.276211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:55.384426 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:55.387546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12449c81-a552-4666-8a48-fd7d92dcc72e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:55.421279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69df81e-2c9b-4675-a407-e1863cc60510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:55.421888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df81e-2c9b-4675-a407-e1863cc60510 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.422184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df81e-2c9b-4675-a407-e1863cc60510 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.422458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df81e-2c9b-4675-a407-e1863cc60510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.488664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69df81e-2c9b-4675-a407-e1863cc60510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:55.490153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69df81e-2c9b-4675-a407-e1863cc60510 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:55.588953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 72/139] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:55.590503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:55.594406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3995d-a671-4422-aa17-3d322b9a5c1b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:55.689646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:55.695965 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3995d-a671-4422-aa17-3d322b9a5c1b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:55.731772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607db1a-5b85-4b5f-b5ff-93bbcdf70715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:55.732480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607db1a-5b85-4b5f-b5ff-93bbcdf70715 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.732814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607db1a-5b85-4b5f-b5ff-93bbcdf70715 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.733133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607db1a-5b85-4b5f-b5ff-93bbcdf70715 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.734284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607db1a-5b85-4b5f-b5ff-93bbcdf70715 None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:55.735761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607db1a-5b85-4b5f-b5ff-93bbcdf70715 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:55.737625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 68/140] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:55 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:55.743494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69df81e-2c9b-4675-a407-e1863cc60510 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:55.843068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69df81e-2c9b-4675-a407-e1863cc60510 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:55.876843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a02be6-3819-45e9-a11b-b0f6c02b41ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:55.877434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02be6-3819-45e9-a11b-b0f6c02b41ff None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.877707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02be6-3819-45e9-a11b-b0f6c02b41ff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.877977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02be6-3819-45e9-a11b-b0f6c02b41ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.878717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a02be6-3819-45e9-a11b-b0f6c02b41ff None admin] RBAC: Adding query filter params (role.id=0e99461723d34ada9c430e01b0c9ae3d, scope.project.id=61e389290239460b8b1a7846922850f0, user.id=8ded85b99b10406890d1e839bb8f875d) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:55.879019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a02be6-3819-45e9-a11b-b0f6c02b41ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:55.880443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a02be6-3819-45e9-a11b-b0f6c02b41ff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:55.886376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 73/141] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:55 2022] GET /identity/v3/role_assignments?user.id=8ded85b99b10406890d1e839bb8f875d&scope.project.id=61e389290239460b8b1a7846922850f0&role.id=0e99461723d34ada9c430e01b0c9ae3d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:56.551664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea60aa0-e49e-4fc8-9ed1-5fb703a9ca85 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.552330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea60aa0-e49e-4fc8-9ed1-5fb703a9ca85 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.552330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea60aa0-e49e-4fc8-9ed1-5fb703a9ca85 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.553641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 69/142] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:56.560683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96ec1b-52f0-4df8-be18-9fc98e5159bf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.561019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96ec1b-52f0-4df8-be18-9fc98e5159bf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.561229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96ec1b-52f0-4df8-be18-9fc98e5159bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.581502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0d96ec1b-52f0-4df8-be18-9fc98e5159bf None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:56.582749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0d96ec1b-52f0-4df8-be18-9fc98e5159bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:56.642202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a1739-a5eb-40fc-988d-c049d6b7b8d9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.642676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a1739-a5eb-40fc-988d-c049d6b7b8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.642960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a1739-a5eb-40fc-988d-c049d6b7b8d9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.644957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 70/143] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:56.645775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:56.650935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266f67e-9968-4ff7-850c-cb97fc58b5f7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.651668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266f67e-9968-4ff7-850c-cb97fc58b5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.651950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266f67e-9968-4ff7-850c-cb97fc58b5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.657786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 74/144] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:56.664346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127d05-9470-4a37-81e9-b1734f3a7abe None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.664630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127d05-9470-4a37-81e9-b1734f3a7abe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.664895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127d05-9470-4a37-81e9-b1734f3a7abe None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.666258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 75/145] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:56.671560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66285010-3fbb-40b8-9079-0e0775bc2c95 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.671843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66285010-3fbb-40b8-9079-0e0775bc2c95 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.672114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66285010-3fbb-40b8-9079-0e0775bc2c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.673558 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a266f67e-9968-4ff7-850c-cb97fc58b5f7 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:56.674895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a266f67e-9968-4ff7-850c-cb97fc58b5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:56.690991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66285010-3fbb-40b8-9079-0e0775bc2c95 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:56.692137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66285010-3fbb-40b8-9079-0e0775bc2c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:56.754003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 71/146] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:56.770131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 76/147] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:56.926227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266f67e-9968-4ff7-850c-cb97fc58b5f7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.015767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66285010-3fbb-40b8-9079-0e0775bc2c95 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.027578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.035986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266f67e-9968-4ff7-850c-cb97fc58b5f7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.069559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7a51a-ac51-46ff-a052-10cff36a6281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.070205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7a51a-ac51-46ff-a052-10cff36a6281 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.070482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7a51a-ac51-46ff-a052-10cff36a6281 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.070750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7a51a-ac51-46ff-a052-10cff36a6281 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.080521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7a51a-ac51-46ff-a052-10cff36a6281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-83d7a51a-ac51-46ff-a052-10cff36a6281 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:57.085235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:57.086170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:57.087446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 72/148] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:57.090969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7a51a-ac51-46ff-a052-10cff36a6281 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.119885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66285010-3fbb-40b8-9079-0e0775bc2c95 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.143105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.154652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96e150-7e90-4daa-bf5b-82a092c842e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.155300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96e150-7e90-4daa-bf5b-82a092c842e5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.155581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96e150-7e90-4daa-bf5b-82a092c842e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.155852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96e150-7e90-4daa-bf5b-82a092c842e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.156963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96e150-7e90-4daa-bf5b-82a092c842e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.158461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96e150-7e90-4daa-bf5b-82a092c842e5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.160030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 77/149] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:57.162210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96e150-7e90-4daa-bf5b-82a092c842e5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.190472 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7a51a-ac51-46ff-a052-10cff36a6281 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.223748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336d0b3-a62a-4daa-91c9-07c6bbe61a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.224381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336d0b3-a62a-4daa-91c9-07c6bbe61a10 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.224663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336d0b3-a62a-4daa-91c9-07c6bbe61a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.224935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336d0b3-a62a-4daa-91c9-07c6bbe61a10 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.235069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336d0b3-a62a-4daa-91c9-07c6bbe61a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7336d0b3-a62a-4daa-91c9-07c6bbe61a10 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:57.239906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:57.240776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:14:57.241691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 73/150] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:57.242682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336d0b3-a62a-4daa-91c9-07c6bbe61a10 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.257834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.263230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96e150-7e90-4daa-bf5b-82a092c842e5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.297787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.298424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.298699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.298975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.299634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:57.299936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.301315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.309568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 78/151] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.310214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.345546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336d0b3-a62a-4daa-91c9-07c6bbe61a10 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.363459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.378051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297ddf-431a-49cb-94d2-fee0ab8779d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.378703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297ddf-431a-49cb-94d2-fee0ab8779d5 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.378985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297ddf-431a-49cb-94d2-fee0ab8779d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.379258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297ddf-431a-49cb-94d2-fee0ab8779d5 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.379961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb297ddf-431a-49cb-94d2-fee0ab8779d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.381534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb297ddf-431a-49cb-94d2-fee0ab8779d5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.403956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 74/152] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:57 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:57.411130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba2e32-d632-4b74-8258-88b30cf66d61 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.411317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297ddf-431a-49cb-94d2-fee0ab8779d5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.451742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.452285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.452557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.452809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.453494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:57.453797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.455245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.465255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 79/153] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.471100 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.496872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.535194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297ddf-431a-49cb-94d2-fee0ab8779d5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.574241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a443e95-5b78-42e9-9273-611b34e7af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.575722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443e95-5b78-42e9-9273-611b34e7af1a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.576212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443e95-5b78-42e9-9273-611b34e7af1a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.576686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443e95-5b78-42e9-9273-611b34e7af1a None admin] PATH_INFO: `/v3/users/34f316f0d17d40288441dff4a57e8cd4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.588943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9377f-eb0a-40e7-8ddb-50b42926849b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.592295 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a443e95-5b78-42e9-9273-611b34e7af1a None admin] RBAC: Authorizing `identity:get_user(user_id=34f316f0d17d40288441dff4a57e8cd4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.593790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a443e95-5b78-42e9-9273-611b34e7af1a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.595508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 75/154] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/users/34f316f0d17d40288441dff4a57e8cd4 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.602548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a443e95-5b78-42e9-9273-611b34e7af1a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.609076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.631349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7aa2-ac23-4fbe-9faf-c94613bc735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.631987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7aa2-ac23-4fbe-9faf-c94613bc735f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.632266 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7aa2-ac23-4fbe-9faf-c94613bc735f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.632537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7aa2-ac23-4fbe-9faf-c94613bc735f None admin] PATH_INFO: `/v3/users/34f316f0d17d40288441dff4a57e8cd4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.633709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b7aa2-ac23-4fbe-9faf-c94613bc735f None admin] RBAC: Authorizing `identity:get_user(user_id=34f316f0d17d40288441dff4a57e8cd4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.635430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b7aa2-ac23-4fbe-9faf-c94613bc735f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.637251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 80/155] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/users/34f316f0d17d40288441dff4a57e8cd4 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:57.642590 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7aa2-ac23-4fbe-9faf-c94613bc735f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.735052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a443e95-5b78-42e9-9273-611b34e7af1a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.763049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7aa2-ac23-4fbe-9faf-c94613bc735f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.773650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d306174-c838-4ac4-975d-540acbaeb2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.774470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d306174-c838-4ac4-975d-540acbaeb2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.774843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d306174-c838-4ac4-975d-540acbaeb2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.775206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d306174-c838-4ac4-975d-540acbaeb2f1 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.776816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d306174-c838-4ac4-975d-540acbaeb2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.778803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d306174-c838-4ac4-975d-540acbaeb2f1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.781135 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.781894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 76/156] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.786665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d306174-c838-4ac4-975d-540acbaeb2f1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.800129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e8e53-c589-4370-be3e-a9641cc4b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.800766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e8e53-c589-4370-be3e-a9641cc4b42a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.801043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e8e53-c589-4370-be3e-a9641cc4b42a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.801312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e8e53-c589-4370-be3e-a9641cc4b42a None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.802568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e8e53-c589-4370-be3e-a9641cc4b42a None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.804130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e8e53-c589-4370-be3e-a9641cc4b42a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.806184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 81/157] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:57.814000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e8e53-c589-4370-be3e-a9641cc4b42a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.887429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.906309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d306174-c838-4ac4-975d-540acbaeb2f1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.948061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a155fe-bdf4-4cb9-af2f-7a17bc2c80fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.948486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a155fe-bdf4-4cb9-af2f-7a17bc2c80fe None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.948768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a155fe-bdf4-4cb9-af2f-7a17bc2c80fe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.949037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a155fe-bdf4-4cb9-af2f-7a17bc2c80fe None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad/users/34f316f0d17d40288441dff4a57e8cd4/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.950988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a155fe-bdf4-4cb9-af2f-7a17bc2c80fe None admin] RBAC: Authorizing `identity:create_grant(project_id=4122f7e8210a46338e361b98e5347dad, user_id=34f316f0d17d40288441dff4a57e8cd4, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.952184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e8e53-c589-4370-be3e-a9641cc4b42a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.952434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a155fe-bdf4-4cb9-af2f-7a17bc2c80fe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.978121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 77/158] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:57 2022] PUT /identity/v3/projects/4122f7e8210a46338e361b98e5347dad/users/34f316f0d17d40288441dff4a57e8cd4/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:57.999078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75921a78-6222-4b07-97b6-52159185ea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.999724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921a78-6222-4b07-97b6-52159185ea6c None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.000002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921a78-6222-4b07-97b6-52159185ea6c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.000270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921a78-6222-4b07-97b6-52159185ea6c None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad/users/34f316f0d17d40288441dff4a57e8cd4/roles/5a2ee20f691f47539f8b1850459be9ca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.002619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75921a78-6222-4b07-97b6-52159185ea6c None admin] RBAC: Authorizing `identity:create_grant(project_id=4122f7e8210a46338e361b98e5347dad, user_id=34f316f0d17d40288441dff4a57e8cd4, role_id=5a2ee20f691f47539f8b1850459be9ca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.004269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75921a78-6222-4b07-97b6-52159185ea6c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.022541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 82/159] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:57 2022] PUT /identity/v3/projects/4122f7e8210a46338e361b98e5347dad/users/34f316f0d17d40288441dff4a57e8cd4/roles/5a2ee20f691f47539f8b1850459be9ca => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:59.700297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6edc1c-f9bb-4d31-863f-bd1f33a92498 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.700828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6edc1c-f9bb-4d31-863f-bd1f33a92498 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.700828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6edc1c-f9bb-4d31-863f-bd1f33a92498 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.702432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 78/160] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:59.708058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8c977-0476-4d9f-abbe-0005a62f54da None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.708463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8c977-0476-4d9f-abbe-0005a62f54da None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.708668 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8c977-0476-4d9f-abbe-0005a62f54da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.727573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66d8c977-0476-4d9f-abbe-0005a62f54da None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:59.728844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66d8c977-0476-4d9f-abbe-0005a62f54da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:59.773552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032e11b-924c-424e-870c-e6ff0ea0ce7a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.773877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032e11b-924c-424e-870c-e6ff0ea0ce7a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.774116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032e11b-924c-424e-870c-e6ff0ea0ce7a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.775492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 79/161] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:59.780739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a25e1-c2db-4f84-a62c-d9ff37ac7cac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.781020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a25e1-c2db-4f84-a62c-d9ff37ac7cac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.781292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a25e1-c2db-4f84-a62c-d9ff37ac7cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.801096 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9c1a25e1-c2db-4f84-a62c-d9ff37ac7cac None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:59.801490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 83/162] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:59.802452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9c1a25e1-c2db-4f84-a62c-d9ff37ac7cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:59.857318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.880570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 80/163] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:00.083933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8c977-0476-4d9f-abbe-0005a62f54da None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.135654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.163540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a25e1-c2db-4f84-a62c-d9ff37ac7cac None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.199737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8c977-0476-4d9f-abbe-0005a62f54da None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.232899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8f434-8384-4693-8cce-b64f1a6347ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.233524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8f434-8384-4693-8cce-b64f1a6347ef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.233799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8f434-8384-4693-8cce-b64f1a6347ef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.234064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8f434-8384-4693-8cce-b64f1a6347ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.243711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8f434-8384-4693-8cce-b64f1a6347ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-80c8f434-8384-4693-8cce-b64f1a6347ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:00.248523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:00.249456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:00.251080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 84/164] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:00.251729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e07d9-688b-42fb-9ebe-f7280baa25ac None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.251729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e07d9-688b-42fb-9ebe-f7280baa25ac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.251849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e07d9-688b-42fb-9ebe-f7280baa25ac None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.254278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 85/165] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:00.254278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948522b-3fb2-4712-8f86-4527414d6a69 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.254480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948522b-3fb2-4712-8f86-4527414d6a69 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.254764 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948522b-3fb2-4712-8f86-4527414d6a69 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.255935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 86/166] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:00.258216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948522b-3fb2-4712-8f86-4527414d6a69 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.266333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a25e1-c2db-4f84-a62c-d9ff37ac7cac None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.300842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe42fb-18ce-4ee4-9f04-e53c629a7659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.301552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe42fb-18ce-4ee4-9f04-e53c629a7659 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.301920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe42fb-18ce-4ee4-9f04-e53c629a7659 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.302303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe42fb-18ce-4ee4-9f04-e53c629a7659 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.303434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe42fb-18ce-4ee4-9f04-e53c629a7659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.304900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe42fb-18ce-4ee4-9f04-e53c629a7659 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.306550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 81/167] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:00.307766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787baa2-0b26-4fad-999a-3b788958f82f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.308007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787baa2-0b26-4fad-999a-3b788958f82f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.308306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787baa2-0b26-4fad-999a-3b788958f82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.325531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4787baa2-0b26-4fad-999a-3b788958f82f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:00.326735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4787baa2-0b26-4fad-999a-3b788958f82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:00.357867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948522b-3fb2-4712-8f86-4527414d6a69 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.359334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.375322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.392447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.392820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 82/168] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:00.393677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.393990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.394119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2b24-17cd-45c0-b11c-a6654059ff7f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.394275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.394377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2b24-17cd-45c0-b11c-a6654059ff7f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.394601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2b24-17cd-45c0-b11c-a6654059ff7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.394951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:00.395250 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.396638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.406783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 87/169] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:00.407395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.417330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-497c2b24-17cd-45c0-b11c-a6654059ff7f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:00.418549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-497c2b24-17cd-45c0-b11c-a6654059ff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:00.488349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 83/170] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:00.489196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2b24-17cd-45c0-b11c-a6654059ff7f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.529897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341772e-71dc-49ba-8a4d-818f1acb4806 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.577511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85143968-319c-448b-b095-875b01bed701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.578970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143968-319c-448b-b095-875b01bed701 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.579566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143968-319c-448b-b095-875b01bed701 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.580035 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143968-319c-448b-b095-875b01bed701 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.581110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85143968-319c-448b-b095-875b01bed701 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.583447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85143968-319c-448b-b095-875b01bed701 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.600204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2b24-17cd-45c0-b11c-a6654059ff7f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.612879 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 88/171] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:00 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:00.626600 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.651363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48e34-f113-4eaf-bf70-c0b43e3dcb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.652400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48e34-f113-4eaf-bf70-c0b43e3dcb76 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.652922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48e34-f113-4eaf-bf70-c0b43e3dcb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.653446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48e34-f113-4eaf-bf70-c0b43e3dcb76 None admin] PATH_INFO: `/v3/users/2f7fa5f3875840d7bd36b3435f601209` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.661935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85143968-319c-448b-b095-875b01bed701 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.676280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48e34-f113-4eaf-bf70-c0b43e3dcb76 None admin] RBAC: Authorizing `identity:get_user(user_id=2f7fa5f3875840d7bd36b3435f601209)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.678510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48e34-f113-4eaf-bf70-c0b43e3dcb76 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.680686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 84/172] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/users/2f7fa5f3875840d7bd36b3435f601209 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:00.686358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48e34-f113-4eaf-bf70-c0b43e3dcb76 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.688378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.782448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85143968-319c-448b-b095-875b01bed701 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.821681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac5dd8-c451-41dc-a745-bf04954c9e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.822429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac5dd8-c451-41dc-a745-bf04954c9e5d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.822710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac5dd8-c451-41dc-a745-bf04954c9e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.822983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac5dd8-c451-41dc-a745-bf04954c9e5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.824981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48e34-f113-4eaf-bf70-c0b43e3dcb76 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.836620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac5dd8-c451-41dc-a745-bf04954c9e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6cac5dd8-c451-41dc-a745-bf04954c9e5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:00.843064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:00.843977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:00.844753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:00.849072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 89/173] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:00.849874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac5dd8-c451-41dc-a745-bf04954c9e5d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.869561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c605d9-d69f-4d85-9207-9093f1a57cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.870231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c605d9-d69f-4d85-9207-9093f1a57cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.870535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c605d9-d69f-4d85-9207-9093f1a57cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.870791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c605d9-d69f-4d85-9207-9093f1a57cd2 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.871927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c605d9-d69f-4d85-9207-9093f1a57cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.873384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c605d9-d69f-4d85-9207-9093f1a57cd2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.875235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 85/174] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:00.878296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c605d9-d69f-4d85-9207-9093f1a57cd2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.933012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.964267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.993958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac5dd8-c451-41dc-a745-bf04954c9e5d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:01.026664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c605d9-d69f-4d85-9207-9093f1a57cd2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:01.030550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0df82-805e-494b-b17c-c32d571da804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:01.031358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0df82-805e-494b-b17c-c32d571da804 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.031788 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0df82-805e-494b-b17c-c32d571da804 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.032188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0df82-805e-494b-b17c-c32d571da804 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.045553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0df82-805e-494b-b17c-c32d571da804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3eb0df82-805e-494b-b17c-c32d571da804 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:01.050612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:01.051609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:01.052519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:01.054756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 90/175] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:01.055650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0df82-805e-494b-b17c-c32d571da804 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:01.073406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:01.074296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.074648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.075032 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.075782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:01.076157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:01.077902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:01.090504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 86/176] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:01.096487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:01.173352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0df82-805e-494b-b17c-c32d571da804 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:01.208629 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d57b-1eae-4253-813a-4bb9586ca672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:01.209586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d57b-1eae-4253-813a-4bb9586ca672 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.210029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d57b-1eae-4253-813a-4bb9586ca672 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.210469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d57b-1eae-4253-813a-4bb9586ca672 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad/users/2f7fa5f3875840d7bd36b3435f601209/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.213143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6d57b-1eae-4253-813a-4bb9586ca672 None admin] RBAC: Authorizing `identity:create_grant(project_id=4122f7e8210a46338e361b98e5347dad, user_id=2f7fa5f3875840d7bd36b3435f601209, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:01.215130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6d57b-1eae-4253-813a-4bb9586ca672 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:01.226667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:01.228095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631410b-6a54-4612-a875-39541e0f749b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:01.230878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:01.240570 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 91/177] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:01 2022] PUT /identity/v3/projects/4122f7e8210a46338e361b98e5347dad/users/2f7fa5f3875840d7bd36b3435f601209/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:01.241822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d57b-1eae-4253-813a-4bb9586ca672 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:01.323582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:01.324320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.324648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.324988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.325730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:01.326110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:01.327656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:01.346277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 87/178] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:01.356645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:01.499207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d57b-1eae-4253-813a-4bb9586ca672 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:01.576021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca38b11-cde5-4d67-9b1b-5496d34cb666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:01.578122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca38b11-cde5-4d67-9b1b-5496d34cb666 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.578623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca38b11-cde5-4d67-9b1b-5496d34cb666 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.580393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca38b11-cde5-4d67-9b1b-5496d34cb666 None admin] PATH_INFO: `/v3/users/34f316f0d17d40288441dff4a57e8cd4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.582807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca38b11-cde5-4d67-9b1b-5496d34cb666 None admin] RBAC: Authorizing `identity:get_user(user_id=34f316f0d17d40288441dff4a57e8cd4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:01.584277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384303d8-47cf-408c-a749-95a9022fb24e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:01.585973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca38b11-cde5-4d67-9b1b-5496d34cb666 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:01.590007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 92/179] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:01 2022] GET /identity/v3/users/34f316f0d17d40288441dff4a57e8cd4 => generated 283 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:01.601780 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca38b11-cde5-4d67-9b1b-5496d34cb666 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:01.663229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0262b0-a5e3-458f-b487-d6f82e9d4c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:01.665194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0262b0-a5e3-458f-b487-d6f82e9d4c9b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.666416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0262b0-a5e3-458f-b487-d6f82e9d4c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.667556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0262b0-a5e3-458f-b487-d6f82e9d4c9b None admin] PATH_INFO: `/v3/users/34f316f0d17d40288441dff4a57e8cd4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.670243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0262b0-a5e3-458f-b487-d6f82e9d4c9b None admin] RBAC: Authorizing `identity:get_user(user_id=34f316f0d17d40288441dff4a57e8cd4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:01.673540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0262b0-a5e3-458f-b487-d6f82e9d4c9b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:01.676589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 88/180] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:01 2022] GET /identity/v3/users/34f316f0d17d40288441dff4a57e8cd4 => generated 283 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:01.682325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0262b0-a5e3-458f-b487-d6f82e9d4c9b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:01.723458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:01.765124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:01.794257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0262b0-a5e3-458f-b487-d6f82e9d4c9b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:01.838315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b261c59-53df-43e0-8d54-aabd798423c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:01.839628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b261c59-53df-43e0-8d54-aabd798423c5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.840055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b261c59-53df-43e0-8d54-aabd798423c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.840455 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b261c59-53df-43e0-8d54-aabd798423c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.862777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca38b11-cde5-4d67-9b1b-5496d34cb666 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:01.951855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515462ee-84b1-4524-9ae3-a4f914dd5b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:01.952736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515462ee-84b1-4524-9ae3-a4f914dd5b40 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.953148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515462ee-84b1-4524-9ae3-a4f914dd5b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.953548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515462ee-84b1-4524-9ae3-a4f914dd5b40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.028186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b261c59-53df-43e0-8d54-aabd798423c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.031133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b261c59-53df-43e0-8d54-aabd798423c5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.084832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515462ee-84b1-4524-9ae3-a4f914dd5b40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.091966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515462ee-84b1-4524-9ae3-a4f914dd5b40 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.095146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.170865 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.191724 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 89/181] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:15:02.197607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b261c59-53df-43e0-8d54-aabd798423c5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.207696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 93/182] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:15:02.214070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515462ee-84b1-4524-9ae3-a4f914dd5b40 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.318625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515462ee-84b1-4524-9ae3-a4f914dd5b40 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.355900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47a825-ed02-4861-9270-88380acf0cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.356705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47a825-ed02-4861-9270-88380acf0cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.357084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47a825-ed02-4861-9270-88380acf0cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.357452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47a825-ed02-4861-9270-88380acf0cb2 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.358714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47a825-ed02-4861-9270-88380acf0cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.360247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47a825-ed02-4861-9270-88380acf0cb2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.362317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 94/183] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.368069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47a825-ed02-4861-9270-88380acf0cb2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.375084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b261c59-53df-43e0-8d54-aabd798423c5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.414496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.456456 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.473796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637301b2-9650-449c-aec4-d7c81a6285b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.474686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637301b2-9650-449c-aec4-d7c81a6285b4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.476056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637301b2-9650-449c-aec4-d7c81a6285b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.477105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637301b2-9650-449c-aec4-d7c81a6285b4 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.479915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47a825-ed02-4861-9270-88380acf0cb2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.480684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637301b2-9650-449c-aec4-d7c81a6285b4 None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.482325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637301b2-9650-449c-aec4-d7c81a6285b4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.485758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 90/184] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:02.493055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637301b2-9650-449c-aec4-d7c81a6285b4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.554810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eac64-8b4f-4077-98f1-83a4fb0510b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.555603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eac64-8b4f-4077-98f1-83a4fb0510b8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.555976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eac64-8b4f-4077-98f1-83a4fb0510b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.556340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eac64-8b4f-4077-98f1-83a4fb0510b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.557225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09eac64-8b4f-4077-98f1-83a4fb0510b8 None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=4122f7e8210a46338e361b98e5347dad, user.id=34f316f0d17d40288441dff4a57e8cd4) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.557692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09eac64-8b4f-4077-98f1-83a4fb0510b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.559257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09eac64-8b4f-4077-98f1-83a4fb0510b8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.574034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 95/185] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/role_assignments?user.id=34f316f0d17d40288441dff4a57e8cd4&scope.project.id=4122f7e8210a46338e361b98e5347dad&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.657669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637301b2-9650-449c-aec4-d7c81a6285b4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.708730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecd219-539a-428a-8c22-00274302b819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.709624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd219-539a-428a-8c22-00274302b819 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.710043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd219-539a-428a-8c22-00274302b819 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.710496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd219-539a-428a-8c22-00274302b819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.711462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecd219-539a-428a-8c22-00274302b819 None admin] RBAC: Adding query filter params (role.id=5a2ee20f691f47539f8b1850459be9ca, scope.project.id=4122f7e8210a46338e361b98e5347dad, user.id=34f316f0d17d40288441dff4a57e8cd4) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.712232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecd219-539a-428a-8c22-00274302b819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.714204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecd219-539a-428a-8c22-00274302b819 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.723556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 91/186] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/role_assignments?user.id=34f316f0d17d40288441dff4a57e8cd4&scope.project.id=4122f7e8210a46338e361b98e5347dad&role.id=5a2ee20f691f47539f8b1850459be9ca => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.099412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f6695-1b63-4f70-b23b-f53fcdc60e0f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.099913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f6695-1b63-4f70-b23b-f53fcdc60e0f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.099985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f6695-1b63-4f70-b23b-f53fcdc60e0f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.101617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 96/187] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:03.107040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a3c9-6832-45db-ae07-04f4dfe7a005 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.107494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a3c9-6832-45db-ae07-04f4dfe7a005 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.107691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a3c9-6832-45db-ae07-04f4dfe7a005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.126076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4f76a3c9-6832-45db-ae07-04f4dfe7a005 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:03.127294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4f76a3c9-6832-45db-ae07-04f4dfe7a005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:03.187726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.189995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df697ac-67c9-41f3-9741-a7da05ed3cec None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.190326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df697ac-67c9-41f3-9741-a7da05ed3cec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.190620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df697ac-67c9-41f3-9741-a7da05ed3cec None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.192017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 97/188] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:03.198523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 92/189] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:03.199795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0b809-cd16-4b35-bc66-bb8a93280519 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.200076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0b809-cd16-4b35-bc66-bb8a93280519 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.200349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0b809-cd16-4b35-bc66-bb8a93280519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.218938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e2c0b809-cd16-4b35-bc66-bb8a93280519 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:03.220145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e2c0b809-cd16-4b35-bc66-bb8a93280519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:03.289197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 98/190] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:03.459428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76a3c9-6832-45db-ae07-04f4dfe7a005 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.559217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0b809-cd16-4b35-bc66-bb8a93280519 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.563170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76a3c9-6832-45db-ae07-04f4dfe7a005 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.592701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.602723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442a656-2a12-4821-a81b-ac9e7be595f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.603873 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442a656-2a12-4821-a81b-ac9e7be595f8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.604448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442a656-2a12-4821-a81b-ac9e7be595f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.604926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442a656-2a12-4821-a81b-ac9e7be595f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.616916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442a656-2a12-4821-a81b-ac9e7be595f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9442a656-2a12-4821-a81b-ac9e7be595f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:03.622448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:03.623309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:03.624116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:03.626596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 93/191] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:03.632673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442a656-2a12-4821-a81b-ac9e7be595f8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.670519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0b809-cd16-4b35-bc66-bb8a93280519 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.688399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.706915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d029-e2aa-4412-bfe2-0a4d946451cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.707547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d029-e2aa-4412-bfe2-0a4d946451cf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.707826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d029-e2aa-4412-bfe2-0a4d946451cf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.708096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d029-e2aa-4412-bfe2-0a4d946451cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.709206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7d029-e2aa-4412-bfe2-0a4d946451cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.710744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7d029-e2aa-4412-bfe2-0a4d946451cf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.713063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 99/192] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.718719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d029-e2aa-4412-bfe2-0a4d946451cf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.739851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442a656-2a12-4821-a81b-ac9e7be595f8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.776329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.777254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.777639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.777894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.778522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:03.778822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.780703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.794407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 94/193] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.795302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.815223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.819758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d029-e2aa-4412-bfe2-0a4d946451cf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.857015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b9e04-6b98-4d64-8cc8-b3a966d95782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.857998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b9e04-6b98-4d64-8cc8-b3a966d95782 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.858305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b9e04-6b98-4d64-8cc8-b3a966d95782 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.858630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b9e04-6b98-4d64-8cc8-b3a966d95782 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.859235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b9e04-6b98-4d64-8cc8-b3a966d95782 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.860766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b9e04-6b98-4d64-8cc8-b3a966d95782 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.886694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 100/194] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:03 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:03.921225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be678a1-7b7f-4e5d-aef6-cb22dd19142c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.940470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.962317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878075d-c40f-4115-ae48-11697c409374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.962983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878075d-c40f-4115-ae48-11697c409374 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.963259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878075d-c40f-4115-ae48-11697c409374 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.963532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878075d-c40f-4115-ae48-11697c409374 None admin] PATH_INFO: `/v3/users/5b849112d0e54daf8796b4f990f5ca21` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.979541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878075d-c40f-4115-ae48-11697c409374 None admin] RBAC: Authorizing `identity:get_user(user_id=5b849112d0e54daf8796b4f990f5ca21)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.981051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878075d-c40f-4115-ae48-11697c409374 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.982035 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a045b3-1248-4a62-85f1-a2d83cb4d5f0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.982360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a045b3-1248-4a62-85f1-a2d83cb4d5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.982609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a045b3-1248-4a62-85f1-a2d83cb4d5f0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.987635 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 95/195] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/users/5b849112d0e54daf8796b4f990f5ca21 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:03.987795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 101/196] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:03.988687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878075d-c40f-4115-ae48-11697c409374 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.990448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececef62-a5e5-47c8-9133-de81b248d8ff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.990728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececef62-a5e5-47c8-9133-de81b248d8ff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.991001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececef62-a5e5-47c8-9133-de81b248d8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.010689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ececef62-a5e5-47c8-9133-de81b248d8ff None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:04.011919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ececef62-a5e5-47c8-9133-de81b248d8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:04.087302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 102/197] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:04.103255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878075d-c40f-4115-ae48-11697c409374 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.140928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed127dc-37c7-4f3e-bee0-6ea0101e9575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.142474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed127dc-37c7-4f3e-bee0-6ea0101e9575 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.142750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed127dc-37c7-4f3e-bee0-6ea0101e9575 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.143016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed127dc-37c7-4f3e-bee0-6ea0101e9575 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.144154 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed127dc-37c7-4f3e-bee0-6ea0101e9575 None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:04.145624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed127dc-37c7-4f3e-bee0-6ea0101e9575 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:04.147310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 96/198] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:04.153499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececef62-a5e5-47c8-9133-de81b248d8ff None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.246562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:04.266950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececef62-a5e5-47c8-9133-de81b248d8ff None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.302923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1f47-7491-4b92-bdf7-c3a10e213ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.303647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1f47-7491-4b92-bdf7-c3a10e213ee1 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.303919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1f47-7491-4b92-bdf7-c3a10e213ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.304187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1f47-7491-4b92-bdf7-c3a10e213ee1 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad/users/5b849112d0e54daf8796b4f990f5ca21/roles/0e99461723d34ada9c430e01b0c9ae3d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.306081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de1f47-7491-4b92-bdf7-c3a10e213ee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4122f7e8210a46338e361b98e5347dad, user_id=5b849112d0e54daf8796b4f990f5ca21, role_id=0e99461723d34ada9c430e01b0c9ae3d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:04.307599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de1f47-7491-4b92-bdf7-c3a10e213ee1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:04.324235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 103/199] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:04 2022] PUT /identity/v3/projects/4122f7e8210a46338e361b98e5347dad/users/5b849112d0e54daf8796b4f990f5ca21/roles/0e99461723d34ada9c430e01b0c9ae3d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:15:04.356334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed127dc-37c7-4f3e-bee0-6ea0101e9575 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.413629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:04.480839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed127dc-37c7-4f3e-bee0-6ea0101e9575 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.520039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae06c6e-4067-4279-b766-43cf5beec5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.520590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae06c6e-4067-4279-b766-43cf5beec5bc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.520866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae06c6e-4067-4279-b766-43cf5beec5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.521129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae06c6e-4067-4279-b766-43cf5beec5bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.531927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae06c6e-4067-4279-b766-43cf5beec5bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-aae06c6e-4067-4279-b766-43cf5beec5bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:04.536727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:04.537634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:04.538502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 97/200] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:04.545568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1f47-7491-4b92-bdf7-c3a10e213ee1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.655162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1f47-7491-4b92-bdf7-c3a10e213ee1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.690433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.691044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.691325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.691598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.692266 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:04.692567 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:04.693948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:04.703576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 104/201] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:04.709266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae06c6e-4067-4279-b766-43cf5beec5bc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.818543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae06c6e-4067-4279-b766-43cf5beec5bc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.820344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:04.859637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349813b0-aac8-4330-adcd-753b2081f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.860889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349813b0-aac8-4330-adcd-753b2081f60d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.861374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349813b0-aac8-4330-adcd-753b2081f60d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.861818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349813b0-aac8-4330-adcd-753b2081f60d None admin] PATH_INFO: `/v3/users/2f7fa5f3875840d7bd36b3435f601209` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.863182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349813b0-aac8-4330-adcd-753b2081f60d None admin] RBAC: Authorizing `identity:get_user(user_id=2f7fa5f3875840d7bd36b3435f601209)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:04.864808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349813b0-aac8-4330-adcd-753b2081f60d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:04.866906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 98/202] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:04 2022] GET /identity/v3/users/2f7fa5f3875840d7bd36b3435f601209 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:04.872876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.883178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:04.988036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012462a-8272-45f7-9437-6cb6afcf65bd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.022723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a1a14-d2e3-46fb-8e9b-cead8c0de9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.023696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a1a14-d2e3-46fb-8e9b-cead8c0de9be None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.024150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a1a14-d2e3-46fb-8e9b-cead8c0de9be None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.024593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a1a14-d2e3-46fb-8e9b-cead8c0de9be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.098976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a1a14-d2e3-46fb-8e9b-cead8c0de9be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.100717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a1a14-d2e3-46fb-8e9b-cead8c0de9be None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.109505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.206360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 105/203] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:05.218142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349813b0-aac8-4330-adcd-753b2081f60d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.318431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349813b0-aac8-4330-adcd-753b2081f60d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.354419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fe0f2-c007-4e1d-a96e-e5734bbadc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.354972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fe0f2-c007-4e1d-a96e-e5734bbadc87 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.355247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fe0f2-c007-4e1d-a96e-e5734bbadc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.355516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fe0f2-c007-4e1d-a96e-e5734bbadc87 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.356648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fe0f2-c007-4e1d-a96e-e5734bbadc87 None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.358121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fe0f2-c007-4e1d-a96e-e5734bbadc87 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.360084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 99/204] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:05.365846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a1a14-d2e3-46fb-8e9b-cead8c0de9be None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.471241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a1a14-d2e3-46fb-8e9b-cead8c0de9be None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.483113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.506876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe3645-7492-41fa-9c30-1c1f69881592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.507474 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3645-7492-41fa-9c30-1c1f69881592 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.507743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3645-7492-41fa-9c30-1c1f69881592 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.508056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3645-7492-41fa-9c30-1c1f69881592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.508775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe3645-7492-41fa-9c30-1c1f69881592 None admin] RBAC: Adding query filter params (role.id=2e66bd66105e4a1e9dcbda045dc2d057, scope.project.id=4122f7e8210a46338e361b98e5347dad, user.id=2f7fa5f3875840d7bd36b3435f601209) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:05.509078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe3645-7492-41fa-9c30-1c1f69881592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.510577 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe3645-7492-41fa-9c30-1c1f69881592 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.516507 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 106/205] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/role_assignments?user.id=2f7fa5f3875840d7bd36b3435f601209&scope.project.id=4122f7e8210a46338e361b98e5347dad&role.id=2e66bd66105e4a1e9dcbda045dc2d057 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:06.206262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed477b1-f2e4-42bf-a947-1f2853b52266 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.206446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed477b1-f2e4-42bf-a947-1f2853b52266 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.206757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed477b1-f2e4-42bf-a947-1f2853b52266 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.208266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 100/206] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:06.213962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22de1f-ee23-426c-bf4b-4b9ecd78d795 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.214520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22de1f-ee23-426c-bf4b-4b9ecd78d795 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.214520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22de1f-ee23-426c-bf4b-4b9ecd78d795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.233501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8c22de1f-ee23-426c-bf4b-4b9ecd78d795 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:06.234703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8c22de1f-ee23-426c-bf4b-4b9ecd78d795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:06.306145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 107/207] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:06.400226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336f6c5-ed77-4034-9b72-9edb95595390 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.400954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336f6c5-ed77-4034-9b72-9edb95595390 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.400954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336f6c5-ed77-4034-9b72-9edb95595390 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.402435 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 101/208] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:06.407686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e2b53-5846-46ea-8d29-27cfd795e0cd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.407949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e2b53-5846-46ea-8d29-27cfd795e0cd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.408234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e2b53-5846-46ea-8d29-27cfd795e0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.426462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2e2e2b53-5846-46ea-8d29-27cfd795e0cd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:06.427632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e2e2b53-5846-46ea-8d29-27cfd795e0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:06.498636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 108/209] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:06.571564 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336f6c5-ed77-4034-9b72-9edb95595390 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.604505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a0920-b901-4050-b8bb-60ee78eb12d0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.604805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a0920-b901-4050-b8bb-60ee78eb12d0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.605073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a0920-b901-4050-b8bb-60ee78eb12d0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.606472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 109/210] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:06.611497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f0ed9-3502-485c-bd69-cc835eada588 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.611778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f0ed9-3502-485c-bd69-cc835eada588 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.612053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f0ed9-3502-485c-bd69-cc835eada588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.614870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:06.630658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d76f0ed9-3502-485c-bd69-cc835eada588 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:06.631826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d76f0ed9-3502-485c-bd69-cc835eada588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:06.656745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:06.684389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336f6c5-ed77-4034-9b72-9edb95595390 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.702166 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 110/211] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:06.719799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87843d-fa80-4e9b-81db-683d052bef04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.720657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87843d-fa80-4e9b-81db-683d052bef04 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.721072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87843d-fa80-4e9b-81db-683d052bef04 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.721565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87843d-fa80-4e9b-81db-683d052bef04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.732270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87843d-fa80-4e9b-81db-683d052bef04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0a87843d-fa80-4e9b-81db-683d052bef04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:06.737491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:06.738460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:06.739263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:06.739953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 102/212] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:06.745618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f0ed9-3502-485c-bd69-cc835eada588 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.767913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87843d-fa80-4e9b-81db-683d052bef04 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.864138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f0ed9-3502-485c-bd69-cc835eada588 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.875442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:06.880428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87843d-fa80-4e9b-81db-683d052bef04 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.908458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.909065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.909338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.909604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.910747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:06.911044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.912424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.921446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 111/213] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:06.922863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c1a1-c45f-42df-9560-32a2d5fdeba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.923497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c1a1-c45f-42df-9560-32a2d5fdeba7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.923771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c1a1-c45f-42df-9560-32a2d5fdeba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.924037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c1a1-c45f-42df-9560-32a2d5fdeba7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.925199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46c1a1-c45f-42df-9560-32a2d5fdeba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.926684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46c1a1-c45f-42df-9560-32a2d5fdeba7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.926931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.928316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 103/214] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:06.933696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c1a1-c45f-42df-9560-32a2d5fdeba7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.952918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.031292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b586b-7632-419c-9b14-67913df65008 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.043583 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c1a1-c45f-42df-9560-32a2d5fdeba7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.064070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420f237-a8c7-4c66-9d1a-226b2102bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.064693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f237-a8c7-4c66-9d1a-226b2102bec7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.064969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f237-a8c7-4c66-9d1a-226b2102bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.065238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f237-a8c7-4c66-9d1a-226b2102bec7 None admin] PATH_INFO: `/v3/users/c514f685ece6490eb5c6f8236fffbc45` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.077069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaadcbd-bd12-418e-9f2b-03bd64c67ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.077678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaadcbd-bd12-418e-9f2b-03bd64c67ce8 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.077953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaadcbd-bd12-418e-9f2b-03bd64c67ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.078066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420f237-a8c7-4c66-9d1a-226b2102bec7 None admin] RBAC: Authorizing `identity:get_user(user_id=c514f685ece6490eb5c6f8236fffbc45)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.078249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaadcbd-bd12-418e-9f2b-03bd64c67ce8 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.078894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaadcbd-bd12-418e-9f2b-03bd64c67ce8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.079568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420f237-a8c7-4c66-9d1a-226b2102bec7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.080313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaadcbd-bd12-418e-9f2b-03bd64c67ce8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.081171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 112/215] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:06 2022] GET /identity/v3/users/c514f685ece6490eb5c6f8236fffbc45 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:07.082003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420f237-a8c7-4c66-9d1a-226b2102bec7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.107869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 104/216] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:06 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:07.108550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaadcbd-bd12-418e-9f2b-03bd64c67ce8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.136007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.186523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.211051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420f237-a8c7-4c66-9d1a-226b2102bec7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.226505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaadcbd-bd12-418e-9f2b-03bd64c67ce8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.249269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca5067-7b0d-4339-b970-6bd88042e662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.249986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca5067-7b0d-4339-b970-6bd88042e662 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.250293 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca5067-7b0d-4339-b970-6bd88042e662 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.250566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca5067-7b0d-4339-b970-6bd88042e662 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.261900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca5067-7b0d-4339-b970-6bd88042e662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:07.264179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.264800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.265085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.265369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.266135 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] RBAC: Adding query filter params (role.id=2e66bd66105e4a1e9dcbda045dc2d057, user.id=c514f685ece6490eb5c6f8236fffbc45, scope.system=all) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:07.266467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-04ca5067-7b0d-4339-b970-6bd88042e662 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:07.266968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:07.267791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:07.268933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:07.269819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.269888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 113/217] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:07.274135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca5067-7b0d-4339-b970-6bd88042e662 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.274699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 105/218] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/role_assignments?user.id=c514f685ece6490eb5c6f8236fffbc45&scope.system=all&role.id=2e66bd66105e4a1e9dcbda045dc2d057 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:07.397401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca5067-7b0d-4339-b970-6bd88042e662 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.438403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.439287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.439707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.440116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.440943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:07.441440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.443038 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.453285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 114/219] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:07.459323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.558894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.575330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08f991-90f0-464f-a13b-edec9e03be66 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.614821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144ae86-4f15-400c-baaa-d8d2b7cef577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.615504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144ae86-4f15-400c-baaa-d8d2b7cef577 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.615778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144ae86-4f15-400c-baaa-d8d2b7cef577 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.616043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144ae86-4f15-400c-baaa-d8d2b7cef577 None admin] PATH_INFO: `/v3/users/5b849112d0e54daf8796b4f990f5ca21` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.617313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144ae86-4f15-400c-baaa-d8d2b7cef577 None admin] RBAC: Authorizing `identity:get_user(user_id=5b849112d0e54daf8796b4f990f5ca21)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.618844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144ae86-4f15-400c-baaa-d8d2b7cef577 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.620764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 106/220] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/users/5b849112d0e54daf8796b4f990f5ca21 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:07.627416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.632349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.739673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e765a-b437-4155-9dae-60b084b25dd2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.775316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb55ae-25df-488e-894b-108506970c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.775871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb55ae-25df-488e-894b-108506970c3c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.776147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb55ae-25df-488e-894b-108506970c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.776418 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb55ae-25df-488e-894b-108506970c3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.849052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb55ae-25df-488e-894b-108506970c3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.851026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb55ae-25df-488e-894b-108506970c3c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.853142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.955165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 115/221] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:07.963569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144ae86-4f15-400c-baaa-d8d2b7cef577 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.069756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144ae86-4f15-400c-baaa-d8d2b7cef577 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.106168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca148b63-0ded-4b11-bce5-500ba669165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.106806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148b63-0ded-4b11-bce5-500ba669165b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.107078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148b63-0ded-4b11-bce5-500ba669165b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.107345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148b63-0ded-4b11-bce5-500ba669165b None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.108450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca148b63-0ded-4b11-bce5-500ba669165b None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.109892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca148b63-0ded-4b11-bce5-500ba669165b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.111541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 107/222] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:08.117588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb55ae-25df-488e-894b-108506970c3c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.223813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb55ae-25df-488e-894b-108506970c3c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.225542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.262011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6fc0b-fcf4-4264-8d43-2b318410aca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.262691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6fc0b-fcf4-4264-8d43-2b318410aca9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.262916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6fc0b-fcf4-4264-8d43-2b318410aca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.263189 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6fc0b-fcf4-4264-8d43-2b318410aca9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.263921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6fc0b-fcf4-4264-8d43-2b318410aca9 None admin] RBAC: Adding query filter params (role.id=0e99461723d34ada9c430e01b0c9ae3d, scope.project.id=4122f7e8210a46338e361b98e5347dad, user.id=5b849112d0e54daf8796b4f990f5ca21) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:08.264224 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6fc0b-fcf4-4264-8d43-2b318410aca9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.265683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6fc0b-fcf4-4264-8d43-2b318410aca9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.272776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 116/223] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/role_assignments?user.id=5b849112d0e54daf8796b4f990f5ca21&scope.project.id=4122f7e8210a46338e361b98e5347dad&role.id=0e99461723d34ada9c430e01b0c9ae3d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.500189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc9786-8660-4acf-8057-f4a458c7d277 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.500746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc9786-8660-4acf-8057-f4a458c7d277 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.500746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc9786-8660-4acf-8057-f4a458c7d277 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.502246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 108/224] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:09.507721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9282e3d-f9b9-40c9-8566-ad29186f7b58 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.508157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9282e3d-f9b9-40c9-8566-ad29186f7b58 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.508242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9282e3d-f9b9-40c9-8566-ad29186f7b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.527902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e9282e3d-f9b9-40c9-8566-ad29186f7b58 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:09.529095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e9282e3d-f9b9-40c9-8566-ad29186f7b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:09.543879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b614-de20-421a-9ee8-5e3f608b7dc4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.544710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b614-de20-421a-9ee8-5e3f608b7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.545249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b614-de20-421a-9ee8-5e3f608b7dc4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.547705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 109/225] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:09.554411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327f2f-1437-432e-a7f4-569ea20a77ac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.555301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327f2f-1437-432e-a7f4-569ea20a77ac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.555790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327f2f-1437-432e-a7f4-569ea20a77ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.582468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1a327f2f-1437-432e-a7f4-569ea20a77ac None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:09.584012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1a327f2f-1437-432e-a7f4-569ea20a77ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:09.606770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 117/226] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:09.607530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.659899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 110/227] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:09.671698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22da45c-1cbf-4970-8287-085b7233bc8d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.671989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22da45c-1cbf-4970-8287-085b7233bc8d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.672260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22da45c-1cbf-4970-8287-085b7233bc8d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.673739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 118/228] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:09.678649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68448f-6402-48ca-be02-6550c452b8db None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.678932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68448f-6402-48ca-be02-6550c452b8db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.679201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68448f-6402-48ca-be02-6550c452b8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.697008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2d68448f-6402-48ca-be02-6550c452b8db None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:09.698244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d68448f-6402-48ca-be02-6550c452b8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:09.766414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 111/229] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:09.870557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22da45c-1cbf-4970-8287-085b7233bc8d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.928485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68448f-6402-48ca-be02-6550c452b8db None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.974923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.983566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22da45c-1cbf-4970-8287-085b7233bc8d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.011892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.018318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae91b-b87b-4e74-9d23-572b83c5203b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.018880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae91b-b87b-4e74-9d23-572b83c5203b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.019165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae91b-b87b-4e74-9d23-572b83c5203b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.019430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae91b-b87b-4e74-9d23-572b83c5203b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.029472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ae91b-b87b-4e74-9d23-572b83c5203b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:10.032162 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68448f-6402-48ca-be02-6550c452b8db None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-431ae91b-b87b-4e74-9d23-572b83c5203b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:10.035067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:10.035944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:10.036720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 119/230] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:10.037381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae91b-b87b-4e74-9d23-572b83c5203b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.067480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb2ec5-78df-45c0-8b85-22f96998247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.068101 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb2ec5-78df-45c0-8b85-22f96998247d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.068375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb2ec5-78df-45c0-8b85-22f96998247d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.068643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb2ec5-78df-45c0-8b85-22f96998247d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.069722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb2ec5-78df-45c0-8b85-22f96998247d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.071202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb2ec5-78df-45c0-8b85-22f96998247d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.073005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 112/231] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:10.073754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb2ec5-78df-45c0-8b85-22f96998247d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.136071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae91b-b87b-4e74-9d23-572b83c5203b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.169028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4251e-f4ef-4fc0-b093-306fc5606f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.169536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb2ec5-78df-45c0-8b85-22f96998247d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.169671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4251e-f4ef-4fc0-b093-306fc5606f4f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.169944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4251e-f4ef-4fc0-b093-306fc5606f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.170240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4251e-f4ef-4fc0-b093-306fc5606f4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.179988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4251e-f4ef-4fc0-b093-306fc5606f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b0e4251e-f4ef-4fc0-b093-306fc5606f4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:10.184956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:10.185950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 120/232] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:10.191421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4251e-f4ef-4fc0-b093-306fc5606f4f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.203247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.203858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.204137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.204409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.205066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:10.205373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.205789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.206810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.216112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 113/233] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:10.216913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.233401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.291553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4251e-f4ef-4fc0-b093-306fc5606f4f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.316315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b96c7-4dd6-41f5-887a-287f35bbee7a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.324680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72ed4-de85-49a2-9d2e-1ac4d628167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.325643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72ed4-de85-49a2-9d2e-1ac4d628167a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.325924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72ed4-de85-49a2-9d2e-1ac4d628167a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.326528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72ed4-de85-49a2-9d2e-1ac4d628167a None admin] PATH_INFO: `/v3/groups` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.327168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f72ed4-de85-49a2-9d2e-1ac4d628167a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.328626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f72ed4-de85-49a2-9d2e-1ac4d628167a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.345856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 121/234] 10.222.0.25 () {60 vars in 1134 bytes} [Wed Nov 2 07:15:10 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:10.348887 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.349507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.349780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.350045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.350740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:10.351040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.352450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.355166 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72ed4-de85-49a2-9d2e-1ac4d628167a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.363113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 114/235] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:10.369168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.454051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.459139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.481499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72ed4-de85-49a2-9d2e-1ac4d628167a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.488444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79628bea-de3f-49a4-a33c-738a17844264 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.517822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289621e-b551-4bae-a7ae-0fee0e1fea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.518961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289621e-b551-4bae-a7ae-0fee0e1fea70 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.519249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289621e-b551-4bae-a7ae-0fee0e1fea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.519517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289621e-b551-4bae-a7ae-0fee0e1fea70 None admin] PATH_INFO: `/v3/users/e56a0ee86b434f4f90b7b904a1066115` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.526568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d1f73-cd37-406c-bbe9-c8a70727772f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.527050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1f73-cd37-406c-bbe9-c8a70727772f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.528066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1f73-cd37-406c-bbe9-c8a70727772f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.528066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1f73-cd37-406c-bbe9-c8a70727772f None admin] PATH_INFO: `/v3/users/c514f685ece6490eb5c6f8236fffbc45` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.529333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d1f73-cd37-406c-bbe9-c8a70727772f None admin] RBAC: Authorizing `identity:get_user(user_id=c514f685ece6490eb5c6f8236fffbc45)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.530112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d1f73-cd37-406c-bbe9-c8a70727772f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.534197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 115/236] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/users/c514f685ece6490eb5c6f8236fffbc45 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:10.537400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d1f73-cd37-406c-bbe9-c8a70727772f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.543277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289621e-b551-4bae-a7ae-0fee0e1fea70 None admin] RBAC: Authorizing `identity:get_user(user_id=e56a0ee86b434f4f90b7b904a1066115)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.543892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289621e-b551-4bae-a7ae-0fee0e1fea70 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.545590 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 122/237] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/users/e56a0ee86b434f4f90b7b904a1066115 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:10.551999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289621e-b551-4bae-a7ae-0fee0e1fea70 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.645242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d1f73-cd37-406c-bbe9-c8a70727772f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.655814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289621e-b551-4bae-a7ae-0fee0e1fea70 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.681889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1093b-cd06-4489-98cf-12f478f792f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.682583 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1093b-cd06-4489-98cf-12f478f792f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.682857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1093b-cd06-4489-98cf-12f478f792f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.683215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1093b-cd06-4489-98cf-12f478f792f0 None admin] PATH_INFO: `/v3/system/users/c514f685ece6490eb5c6f8236fffbc45/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.684506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1093b-cd06-4489-98cf-12f478f792f0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c514f685ece6490eb5c6f8236fffbc45, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.685876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1093b-cd06-4489-98cf-12f478f792f0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.694836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1a8f7-c9c8-4d74-a6a1-3b49ed0deae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.695576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1a8f7-c9c8-4d74-a6a1-3b49ed0deae1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.695878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1a8f7-c9c8-4d74-a6a1-3b49ed0deae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.696199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1a8f7-c9c8-4d74-a6a1-3b49ed0deae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.696928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1a8f7-c9c8-4d74-a6a1-3b49ed0deae1 None admin] RBAC: Adding query filter params (role.id=0e99461723d34ada9c430e01b0c9ae3d, user.id=e56a0ee86b434f4f90b7b904a1066115, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:10.697246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1a8f7-c9c8-4d74-a6a1-3b49ed0deae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.697739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 116/238] 10.222.0.25 () {58 vars in 1334 bytes} [Wed Nov 2 07:15:10 2022] PUT /identity/v3/system/users/c514f685ece6490eb5c6f8236fffbc45/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:15:10.698837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1a8f7-c9c8-4d74-a6a1-3b49ed0deae1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.700970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.714589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 123/239] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/role_assignments?user.id=e56a0ee86b434f4f90b7b904a1066115&scope.system=all&role.id=0e99461723d34ada9c430e01b0c9ae3d => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:12.515611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8912480-eb9c-4bec-a6c4-38a0d7bc984e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.516142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8912480-eb9c-4bec-a6c4-38a0d7bc984e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.516217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8912480-eb9c-4bec-a6c4-38a0d7bc984e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.517575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 117/240] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:12.523015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8204070-074a-4314-b7d3-9fb398a6f9f0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.523455 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8204070-074a-4314-b7d3-9fb398a6f9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.523526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8204070-074a-4314-b7d3-9fb398a6f9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.542925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f8204070-074a-4314-b7d3-9fb398a6f9f0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:12.544124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f8204070-074a-4314-b7d3-9fb398a6f9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:12.613970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 124/241] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:12.854953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e317b-5e31-4f04-b7e4-672d13f05abc None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.855132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224e203-adad-46d2-8210-413a2768fade None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.855207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e317b-5e31-4f04-b7e4-672d13f05abc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.855498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224e203-adad-46d2-8210-413a2768fade None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.855683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e317b-5e31-4f04-b7e4-672d13f05abc None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.855755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224e203-adad-46d2-8210-413a2768fade None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.856896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 118/242] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:12.858141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 125/243] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:12.861759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fab2a-2b60-4cb4-9654-144897112256 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.862069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eefd1-db55-4594-ac4b-87fb39af9762 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.862250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fab2a-2b60-4cb4-9654-144897112256 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.862414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eefd1-db55-4594-ac4b-87fb39af9762 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.862513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fab2a-2b60-4cb4-9654-144897112256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.862663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eefd1-db55-4594-ac4b-87fb39af9762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.875906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.882344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bd7fab2a-2b60-4cb4-9654-144897112256 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:12.882719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f80eefd1-db55-4594-ac4b-87fb39af9762 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:12.883892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f80eefd1-db55-4594-ac4b-87fb39af9762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:12.884036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bd7fab2a-2b60-4cb4-9654-144897112256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:12.951424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 119/244] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:12.952278 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fab2a-2b60-4cb4-9654-144897112256 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.956304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 126/245] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:13.052669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fab2a-2b60-4cb4-9654-144897112256 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.087090 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1f22d-5a4e-4275-9031-6330cc3fad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.087775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1f22d-5a4e-4275-9031-6330cc3fad93 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.088050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1f22d-5a4e-4275-9031-6330cc3fad93 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.088315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1f22d-5a4e-4275-9031-6330cc3fad93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.089427 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1f22d-5a4e-4275-9031-6330cc3fad93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.090938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1f22d-5a4e-4275-9031-6330cc3fad93 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.094133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 120/246] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.098251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80eefd1-db55-4594-ac4b-87fb39af9762 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.155697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.198644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80eefd1-db55-4594-ac4b-87fb39af9762 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.233789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1f22d-5a4e-4275-9031-6330cc3fad93 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.234111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057431e-cb34-4f76-b0f4-7cf02093cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.234738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057431e-cb34-4f76-b0f4-7cf02093cc04 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.234995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057431e-cb34-4f76-b0f4-7cf02093cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.235264 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057431e-cb34-4f76-b0f4-7cf02093cc04 None admin] PATH_INFO: `/v3/groups` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.235886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057431e-cb34-4f76-b0f4-7cf02093cc04 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.237330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057431e-cb34-4f76-b0f4-7cf02093cc04 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.239283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.258281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 127/247] 10.222.0.25 () {60 vars in 1134 bytes} [Wed Nov 2 07:15:13 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:13.260176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057431e-cb34-4f76-b0f4-7cf02093cc04 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.352105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1f22d-5a4e-4275-9031-6330cc3fad93 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.372569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057431e-cb34-4f76-b0f4-7cf02093cc04 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.390346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7aedc-67f7-45b9-8ce4-41634755ee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.390999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7aedc-67f7-45b9-8ce4-41634755ee2b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.391105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.391286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7aedc-67f7-45b9-8ce4-41634755ee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.391541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7aedc-67f7-45b9-8ce4-41634755ee2b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.405111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7aedc-67f7-45b9-8ce4-41634755ee2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-90c7aedc-67f7-45b9-8ce4-41634755ee2b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:13.411415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:13.412339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:13.413134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 121/248] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:13.418920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4d510-259c-4434-bd06-f1918219d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.419225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7aedc-67f7-45b9-8ce4-41634755ee2b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.419600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4d510-259c-4434-bd06-f1918219d768 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.419882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4d510-259c-4434-bd06-f1918219d768 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.420152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4d510-259c-4434-bd06-f1918219d768 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.432933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4d510-259c-4434-bd06-f1918219d768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-24f4d510-259c-4434-bd06-f1918219d768 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:13.438489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:13.439337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:13.441337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:13.443840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 128/249] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:13.444732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4d510-259c-4434-bd06-f1918219d768 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.493828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.551061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7aedc-67f7-45b9-8ce4-41634755ee2b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.592759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4d510-259c-4434-bd06-f1918219d768 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.598829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.599441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.599719 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.599995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.600659 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:13.600963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.602531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.614367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 122/250] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:13.619595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.644916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.646307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.646625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.646863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.647738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:13.648044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.650069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.662078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 129/251] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.675137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.733223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.761201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902aed7-a85a-42ae-a19d-517dbc5effda None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.797014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.810909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa06576-2d1e-4b6f-afbc-9d47f3b90adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.811941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa06576-2d1e-4b6f-afbc-9d47f3b90adc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.812450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa06576-2d1e-4b6f-afbc-9d47f3b90adc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.812900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa06576-2d1e-4b6f-afbc-9d47f3b90adc None admin] PATH_INFO: `/v3/users/e56a0ee86b434f4f90b7b904a1066115` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.814228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa06576-2d1e-4b6f-afbc-9d47f3b90adc None admin] RBAC: Authorizing `identity:get_user(user_id=e56a0ee86b434f4f90b7b904a1066115)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.815407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79d995-0e9e-4135-b9f7-0e3d9d9f20af None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.815821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa06576-2d1e-4b6f-afbc-9d47f3b90adc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.818161 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 123/252] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/users/e56a0ee86b434f4f90b7b904a1066115 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.828663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa06576-2d1e-4b6f-afbc-9d47f3b90adc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.865654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8095-8910-4e3d-a567-699d8cbe30cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.866328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8095-8910-4e3d-a567-699d8cbe30cc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.866773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8095-8910-4e3d-a567-699d8cbe30cc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.867230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8095-8910-4e3d-a567-699d8cbe30cc None admin] PATH_INFO: `/v3/users/c514f685ece6490eb5c6f8236fffbc45` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.868381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a8095-8910-4e3d-a567-699d8cbe30cc None admin] RBAC: Authorizing `identity:get_user(user_id=c514f685ece6490eb5c6f8236fffbc45)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.870191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a8095-8910-4e3d-a567-699d8cbe30cc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.872877 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 130/253] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/users/c514f685ece6490eb5c6f8236fffbc45 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.879491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8095-8910-4e3d-a567-699d8cbe30cc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.960953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa06576-2d1e-4b6f-afbc-9d47f3b90adc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.996969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f3af2-3626-4168-9d56-ec4631a3af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.997959 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3af2-3626-4168-9d56-ec4631a3af73 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.998451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3af2-3626-4168-9d56-ec4631a3af73 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.998912 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3af2-3626-4168-9d56-ec4631a3af73 None admin] PATH_INFO: `/v3/system/users/e56a0ee86b434f4f90b7b904a1066115/roles/0e99461723d34ada9c430e01b0c9ae3d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.000585 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f3af2-3626-4168-9d56-ec4631a3af73 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e56a0ee86b434f4f90b7b904a1066115, role_id=0e99461723d34ada9c430e01b0c9ae3d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:14.002228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f3af2-3626-4168-9d56-ec4631a3af73 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:14.013637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8095-8910-4e3d-a567-699d8cbe30cc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:14.013913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 124/254] 10.222.0.25 () {58 vars in 1334 bytes} [Wed Nov 2 07:15:13 2022] PUT /identity/v3/system/users/e56a0ee86b434f4f90b7b904a1066115/roles/0e99461723d34ada9c430e01b0c9ae3d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:14.028891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:14.063362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90339cb-657a-40a7-a464-9755797ea66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:14.064299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90339cb-657a-40a7-a464-9755797ea66b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.064783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90339cb-657a-40a7-a464-9755797ea66b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.067574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90339cb-657a-40a7-a464-9755797ea66b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.068561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90339cb-657a-40a7-a464-9755797ea66b None admin] RBAC: Adding query filter params (role.id=2e66bd66105e4a1e9dcbda045dc2d057, user.id=c514f685ece6490eb5c6f8236fffbc45, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:14.069015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90339cb-657a-40a7-a464-9755797ea66b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:14.070807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90339cb-657a-40a7-a464-9755797ea66b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:14.087062 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 131/255] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/role_assignments?user.id=c514f685ece6490eb5c6f8236fffbc45&scope.system=all&role.id=2e66bd66105e4a1e9dcbda045dc2d057 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:15.739944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ee95f-f9aa-406e-a58b-13e8ea15a848 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.740986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ee95f-f9aa-406e-a58b-13e8ea15a848 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.741569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ee95f-f9aa-406e-a58b-13e8ea15a848 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.743649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 125/256] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:15.749065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f360b-6b64-4311-a8b8-6f055e825c3d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.750005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f360b-6b64-4311-a8b8-6f055e825c3d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.750496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f360b-6b64-4311-a8b8-6f055e825c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.779173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-689f360b-6b64-4311-a8b8-6f055e825c3d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:15.780740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-689f360b-6b64-4311-a8b8-6f055e825c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:15.852662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 132/257] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:15:15.883661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27267236-aae7-455b-9020-a852cf7fab30 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.884302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27267236-aae7-455b-9020-a852cf7fab30 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.885006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27267236-aae7-455b-9020-a852cf7fab30 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.886615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 126/258] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:15.892083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fcab0-8468-4b75-ae75-4460fd27fa99 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.892626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fcab0-8468-4b75-ae75-4460fd27fa99 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.893081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fcab0-8468-4b75-ae75-4460fd27fa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.941957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c81fcab0-8468-4b75-ae75-4460fd27fa99 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:15.943534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c81fcab0-8468-4b75-ae75-4460fd27fa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:15.951549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 133/259] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:15:16.402564 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27267236-aae7-455b-9020-a852cf7fab30 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.419723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27267236-aae7-455b-9020-a852cf7fab30 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.424419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b4802-1df3-4c40-8127-223b3e56ba44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.425266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4802-1df3-4c40-8127-223b3e56ba44 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.425729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4802-1df3-4c40-8127-223b3e56ba44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.426209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4802-1df3-4c40-8127-223b3e56ba44 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.441496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b4802-1df3-4c40-8127-223b3e56ba44 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4a8b4802-1df3-4c40-8127-223b3e56ba44 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:16.446598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:16.447910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:16.448692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:16.450991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 127/260] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:16.456907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fcab0-8468-4b75-ae75-4460fd27fa99 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.467275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fcab0-8468-4b75-ae75-4460fd27fa99 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.474163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.475045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.475523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.475625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f0a2c-5112-4525-bb96-198a44d61bd8 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.475854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f0a2c-5112-4525-bb96-198a44d61bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.476128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f0a2c-5112-4525-bb96-198a44d61bd8 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.476518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.477423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:16.478884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 128/261] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:16.479380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.481411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:16.481411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:15:16.481411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:15:16.482714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.489895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0395-27a8-43bd-9d17-682ee197ec16 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.490187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0395-27a8-43bd-9d17-682ee197ec16 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.490457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0395-27a8-43bd-9d17-682ee197ec16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.496375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 134/262] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.501812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.521664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa998e-ae68-46ba-b267-45481e37799f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.523529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-482a0395-27a8-43bd-9d17-682ee197ec16 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:16.525713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-482a0395-27a8-43bd-9d17-682ee197ec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:16.530789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.531657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.532163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.532646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] PATH_INFO: `/v3/groups/97ee3470da97486fb7229f98f17fc652` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.544111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 129/263] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:15:16.551830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0395-27a8-43bd-9d17-682ee197ec16 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.552673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] RBAC: Authorizing `identity:get_group(group_id=97ee3470da97486fb7229f98f17fc652)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.553918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:16.553918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:15:16.553918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:15:16.555739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.562338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 135/264] 10.222.0.25 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/groups/97ee3470da97486fb7229f98f17fc652 => generated 226 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.569238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.574951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0395-27a8-43bd-9d17-682ee197ec16 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.586346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26386a8-9908-47a5-892c-23bfd8b10bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.588492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26386a8-9908-47a5-892c-23bfd8b10bff admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.588970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26386a8-9908-47a5-892c-23bfd8b10bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.589437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26386a8-9908-47a5-892c-23bfd8b10bff admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.601576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d8001-6f6d-4d41-bf0e-8440efb64402 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.606322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26386a8-9908-47a5-892c-23bfd8b10bff admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f26386a8-9908-47a5-892c-23bfd8b10bff admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:16.611690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:16.612575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:16.613358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:16.615661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 130/265] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:16.616494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e57068-af46-44c9-bfc4-85c6d9e4e9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.620246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e57068-af46-44c9-bfc4-85c6d9e4e9fe admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.620246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e57068-af46-44c9-bfc4-85c6d9e4e9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.620246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e57068-af46-44c9-bfc4-85c6d9e4e9fe admin admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.621618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26386a8-9908-47a5-892c-23bfd8b10bff admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.622852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e57068-af46-44c9-bfc4-85c6d9e4e9fe admin admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.624926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e57068-af46-44c9-bfc4-85c6d9e4e9fe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.627557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 136/266] 10.222.0.25 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.632463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e57068-af46-44c9-bfc4-85c6d9e4e9fe admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.637008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26386a8-9908-47a5-892c-23bfd8b10bff admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.642438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.643372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.643842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.644347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.645203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:16.645759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.647898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:16.647898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:15:16.647898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: Nov 02 07:15:16.649267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.661129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e57068-af46-44c9-bfc4-85c6d9e4e9fe admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.662820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 131/267] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.668194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.680820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.681568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.681845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.682130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0/groups/97ee3470da97486fb7229f98f17fc652/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.687395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e389290239460b8b1a7846922850f0, group_id=97ee3470da97486fb7229f98f17fc652, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.687873 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92cec6-2a34-4f24-85e0-53a04844b6a1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.688522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:16.688522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:15:16.688522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:15:16.690469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.710837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.711495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.712123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.712539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] PATH_INFO: `/v3/groups/97ee3470da97486fb7229f98f17fc652` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.714036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] RBAC: Authorizing `identity:get_group(group_id=97ee3470da97486fb7229f98f17fc652)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.715456 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:16.715456 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:15:16.715456 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: Nov 02 07:15:16.716908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.718831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 132/268] 10.222.0.25 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/groups/97ee3470da97486fb7229f98f17fc652 => generated 226 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.724486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.725722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 137/269] 10.222.0.25 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:16 2022] PUT /identity/v3/projects/61e389290239460b8b1a7846922850f0/groups/97ee3470da97486fb7229f98f17fc652/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:16.735785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa18c3d-4ade-437d-a100-c160a30fda7a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.758124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:16.821961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6d9f-bfb4-4e3a-ac14-8f224fb0ee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.822925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6d9f-bfb4-4e3a-ac14-8f224fb0ee10 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.822925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6d9f-bfb4-4e3a-ac14-8f224fb0ee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.823280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6d9f-bfb4-4e3a-ac14-8f224fb0ee10 admin admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.824306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6d9f-bfb4-4e3a-ac14-8f224fb0ee10 admin admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.826478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6d9f-bfb4-4e3a-ac14-8f224fb0ee10 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.828590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 133/270] 10.222.0.25 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:16.834331 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.858113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae3e8d-bafc-4ee3-b448-fea2af22ad90 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.868236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86486c-e711-4408-99ec-2b98c06ba0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.869133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86486c-e711-4408-99ec-2b98c06ba0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.869745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86486c-e711-4408-99ec-2b98c06ba0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.869935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86486c-e711-4408-99ec-2b98c06ba0f1 admin admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0/groups/97ee3470da97486fb7229f98f17fc652/roles/5a2ee20f691f47539f8b1850459be9ca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.875833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86486c-e711-4408-99ec-2b98c06ba0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e389290239460b8b1a7846922850f0, group_id=97ee3470da97486fb7229f98f17fc652, role_id=5a2ee20f691f47539f8b1850459be9ca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.878522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86486c-e711-4408-99ec-2b98c06ba0f1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.912775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 138/271] 10.222.0.25 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:16 2022] PUT /identity/v3/projects/61e389290239460b8b1a7846922850f0/groups/97ee3470da97486fb7229f98f17fc652/roles/5a2ee20f691f47539f8b1850459be9ca => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:17.008725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6d9f-bfb4-4e3a-ac14-8f224fb0ee10 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:17.019285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6d9f-bfb4-4e3a-ac14-8f224fb0ee10 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:17.097438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798113b-3cad-49f7-9d84-423ae8b000e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:17.098079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798113b-3cad-49f7-9d84-423ae8b000e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.098386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798113b-3cad-49f7-9d84-423ae8b000e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.098658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798113b-3cad-49f7-9d84-423ae8b000e7 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.115830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798113b-3cad-49f7-9d84-423ae8b000e7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d798113b-3cad-49f7-9d84-423ae8b000e7 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:17.122419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:17.124667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:17.126722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 134/272] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 116 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:17.131306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86486c-e711-4408-99ec-2b98c06ba0f1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:17.152919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86486c-e711-4408-99ec-2b98c06ba0f1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:17.164423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:17.164967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.165244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.165892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.166611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:17.166890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:17.168498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:17.182653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 139/273] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:17.187903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798113b-3cad-49f7-9d84-423ae8b000e7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:17.204093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798113b-3cad-49f7-9d84-423ae8b000e7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:17.208659 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f707319-d532-479e-a414-684624ae3cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:17.209304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f707319-d532-479e-a414-684624ae3cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.209577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f707319-d532-479e-a414-684624ae3cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.209846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f707319-d532-479e-a414-684624ae3cb2 admin admin] PATH_INFO: `/v3/groups/97ee3470da97486fb7229f98f17fc652` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.215026 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f707319-d532-479e-a414-684624ae3cb2 admin admin] RBAC: Authorizing `identity:get_group(group_id=97ee3470da97486fb7229f98f17fc652)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:17.217051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f707319-d532-479e-a414-684624ae3cb2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:17.218768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 135/274] 10.222.0.25 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:17 2022] GET /identity/v3/groups/97ee3470da97486fb7229f98f17fc652 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:17.224379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:17.234956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acb9a8-14f1-4b44-908e-5d1d0f204922 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:17.239447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925925e7-f5ea-407e-9479-b8ba4e4c14f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:17.240078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925925e7-f5ea-407e-9479-b8ba4e4c14f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.240377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925925e7-f5ea-407e-9479-b8ba4e4c14f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.240625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925925e7-f5ea-407e-9479-b8ba4e4c14f2 admin admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.241765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925925e7-f5ea-407e-9479-b8ba4e4c14f2 admin admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:17.244460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925925e7-f5ea-407e-9479-b8ba4e4c14f2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:17.246580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 140/275] 10.222.0.25 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:17 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:17.252094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f707319-d532-479e-a414-684624ae3cb2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:17.267267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f707319-d532-479e-a414-684624ae3cb2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:17.278722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e298c4-d8d9-4df8-a7d6-3f84fb19346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:17.279409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e298c4-d8d9-4df8-a7d6-3f84fb19346b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.279682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e298c4-d8d9-4df8-a7d6-3f84fb19346b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.279950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e298c4-d8d9-4df8-a7d6-3f84fb19346b admin admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad/groups/97ee3470da97486fb7229f98f17fc652/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.287810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e298c4-d8d9-4df8-a7d6-3f84fb19346b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4122f7e8210a46338e361b98e5347dad, group_id=97ee3470da97486fb7229f98f17fc652, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:17.288870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-19e298c4-d8d9-4df8-a7d6-3f84fb19346b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:17.288870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:15:17.288870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: Nov 02 07:15:17.291255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e298c4-d8d9-4df8-a7d6-3f84fb19346b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:17.310367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 136/276] 10.222.0.25 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:17 2022] PUT /identity/v3/projects/4122f7e8210a46338e361b98e5347dad/groups/97ee3470da97486fb7229f98f17fc652/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:17.442890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fb90b-bf6b-4cc8-b56e-6ba8ffc2066a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.443183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fb90b-bf6b-4cc8-b56e-6ba8ffc2066a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.443355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fb90b-bf6b-4cc8-b56e-6ba8ffc2066a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.444846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 141/277] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:17.451517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa95d8d-ed59-427d-9943-9b78a690e6a9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.451798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa95d8d-ed59-427d-9943-9b78a690e6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.452050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa95d8d-ed59-427d-9943-9b78a690e6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.479301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-aaa95d8d-ed59-427d-9943-9b78a690e6a9 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:17.480534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aaa95d8d-ed59-427d-9943-9b78a690e6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:17.563963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 137/278] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:17.923361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fb90b-bf6b-4cc8-b56e-6ba8ffc2066a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.006380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.072885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fb90b-bf6b-4cc8-b56e-6ba8ffc2066a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.118356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2025946-5f30-44f8-be03-d2abb903c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.119158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2025946-5f30-44f8-be03-d2abb903c812 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.119632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2025946-5f30-44f8-be03-d2abb903c812 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.119903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2025946-5f30-44f8-be03-d2abb903c812 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.131525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2025946-5f30-44f8-be03-d2abb903c812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e2025946-5f30-44f8-be03-d2abb903c812 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:18.136710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:18.137597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:18.139047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 142/279] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:18.143723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa95d8d-ed59-427d-9943-9b78a690e6a9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.172549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211633-d27c-4833-b8a7-061852885944 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.173023 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211633-d27c-4833-b8a7-061852885944 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.173302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211633-d27c-4833-b8a7-061852885944 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.174605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.176725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 143/280] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:18.183558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eaf08-8879-4940-a2c5-91eeffc2913a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.183836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eaf08-8879-4940-a2c5-91eeffc2913a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.184106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eaf08-8879-4940-a2c5-91eeffc2913a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.218858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6c7eaf08-8879-4940-a2c5-91eeffc2913a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:18.220459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6c7eaf08-8879-4940-a2c5-91eeffc2913a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:18.313010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 144/281] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:15:18.327932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa95d8d-ed59-427d-9943-9b78a690e6a9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.383406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.384022 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.384327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.384592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.385354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:18.385656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.387063 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.396263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 138/282] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.402005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eaf08-8879-4940-a2c5-91eeffc2913a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.458759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.509532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eaf08-8879-4940-a2c5-91eeffc2913a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.561551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132f88a-e8a3-48b3-8a30-b3f1e57b0578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.562297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132f88a-e8a3-48b3-8a30-b3f1e57b0578 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.562559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132f88a-e8a3-48b3-8a30-b3f1e57b0578 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.562873 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132f88a-e8a3-48b3-8a30-b3f1e57b0578 None admin] PATH_INFO: `/v3/users/e56a0ee86b434f4f90b7b904a1066115` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.564154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132f88a-e8a3-48b3-8a30-b3f1e57b0578 None admin] RBAC: Authorizing `identity:get_user(user_id=e56a0ee86b434f4f90b7b904a1066115)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.565646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132f88a-e8a3-48b3-8a30-b3f1e57b0578 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.567665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 145/283] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/users/e56a0ee86b434f4f90b7b904a1066115 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.573590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.664615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.688506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ded66-f7b6-47e0-b4ee-fe1e75bb2e9e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.730579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb6adb-d358-40e5-9771-9eca54c9a961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.731124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb6adb-d358-40e5-9771-9eca54c9a961 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.731398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb6adb-d358-40e5-9771-9eca54c9a961 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.731665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb6adb-d358-40e5-9771-9eca54c9a961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.732401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb6adb-d358-40e5-9771-9eca54c9a961 None admin] RBAC: Adding query filter params (role.id=0e99461723d34ada9c430e01b0c9ae3d, user.id=e56a0ee86b434f4f90b7b904a1066115, scope.system=all) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:18.732705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb6adb-d358-40e5-9771-9eca54c9a961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.734188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb6adb-d358-40e5-9771-9eca54c9a961 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.753342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 139/284] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/role_assignments?user.id=e56a0ee86b434f4f90b7b904a1066115&scope.system=all&role.id=0e99461723d34ada9c430e01b0c9ae3d => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.777663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcff97-ce9b-4586-a9bb-3fadeac17cc7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.777990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcff97-ce9b-4586-a9bb-3fadeac17cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.778238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcff97-ce9b-4586-a9bb-3fadeac17cc7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.779805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 146/285] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:18.785212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85990a6-56a9-452a-94c0-f002ade8f1f1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.785489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85990a6-56a9-452a-94c0-f002ade8f1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.785761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85990a6-56a9-452a-94c0-f002ade8f1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.799811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcff97-ce9b-4586-a9bb-3fadeac17cc7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.815482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f85990a6-56a9-452a-94c0-f002ade8f1f1 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:18.816684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f85990a6-56a9-452a-94c0-f002ade8f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:18.823247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcff97-ce9b-4586-a9bb-3fadeac17cc7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.828683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 140/286] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:15:18.831337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0df4d-2a08-41f8-8934-615e9c2a3b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.831992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0df4d-2a08-41f8-8934-615e9c2a3b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.832336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0df4d-2a08-41f8-8934-615e9c2a3b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.832612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0df4d-2a08-41f8-8934-615e9c2a3b8f admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.856138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0df4d-2a08-41f8-8934-615e9c2a3b8f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2be0df4d-2a08-41f8-8934-615e9c2a3b8f admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:18.861213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:18.862168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:18.863417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 147/287] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:18.870861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85990a6-56a9-452a-94c0-f002ade8f1f1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.888902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85990a6-56a9-452a-94c0-f002ade8f1f1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.897321 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.897952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.898257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.898528 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.899200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:18.899502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.901190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.917704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 141/288] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.923527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0df4d-2a08-41f8-8934-615e9c2a3b8f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.936229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0df4d-2a08-41f8-8934-615e9c2a3b8f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.947658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab29882-ef8f-4d93-93d7-ecd0f8913ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.948368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29882-ef8f-4d93-93d7-ecd0f8913ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.948640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29882-ef8f-4d93-93d7-ecd0f8913ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.948908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29882-ef8f-4d93-93d7-ecd0f8913ca7 admin admin] PATH_INFO: `/v3/groups/97ee3470da97486fb7229f98f17fc652` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.950771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab29882-ef8f-4d93-93d7-ecd0f8913ca7 admin admin] RBAC: Authorizing `identity:get_group(group_id=97ee3470da97486fb7229f98f17fc652)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.952820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab29882-ef8f-4d93-93d7-ecd0f8913ca7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.954900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 148/289] 10.222.0.25 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/groups/97ee3470da97486fb7229f98f17fc652 => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.960607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.975467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec40f2-60ec-4293-9d79-93458115dcdb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.985404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381127a5-bb37-438e-a32d-6f0a15cd2957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.985982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381127a5-bb37-438e-a32d-6f0a15cd2957 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.986304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381127a5-bb37-438e-a32d-6f0a15cd2957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.986581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381127a5-bb37-438e-a32d-6f0a15cd2957 admin admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.990884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381127a5-bb37-438e-a32d-6f0a15cd2957 admin admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.992988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381127a5-bb37-438e-a32d-6f0a15cd2957 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.995317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 142/290] 10.222.0.25 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:19.001315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab29882-ef8f-4d93-93d7-ecd0f8913ca7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.016400 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab29882-ef8f-4d93-93d7-ecd0f8913ca7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.023404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4bfeb-8bd7-48af-9b0e-46e5d4575ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.024127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4bfeb-8bd7-48af-9b0e-46e5d4575ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.024401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4bfeb-8bd7-48af-9b0e-46e5d4575ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.024669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4bfeb-8bd7-48af-9b0e-46e5d4575ecd admin admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad/groups/97ee3470da97486fb7229f98f17fc652/roles/5a2ee20f691f47539f8b1850459be9ca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.032257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4bfeb-8bd7-48af-9b0e-46e5d4575ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4122f7e8210a46338e361b98e5347dad, group_id=97ee3470da97486fb7229f98f17fc652, role_id=5a2ee20f691f47539f8b1850459be9ca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.035042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4bfeb-8bd7-48af-9b0e-46e5d4575ecd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.052539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 149/291] 10.222.0.25 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:19 2022] PUT /identity/v3/projects/4122f7e8210a46338e361b98e5347dad/groups/97ee3470da97486fb7229f98f17fc652/roles/5a2ee20f691f47539f8b1850459be9ca => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:19.314977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381127a5-bb37-438e-a32d-6f0a15cd2957 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.325800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381127a5-bb37-438e-a32d-6f0a15cd2957 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.389263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7dfd8a-89bd-4b4b-abc0-e3abf98835a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.389901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7dfd8a-89bd-4b4b-abc0-e3abf98835a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.390206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7dfd8a-89bd-4b4b-abc0-e3abf98835a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.390480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7dfd8a-89bd-4b4b-abc0-e3abf98835a9 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.403313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7dfd8a-89bd-4b4b-abc0-e3abf98835a9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5a7dfd8a-89bd-4b4b-abc0-e3abf98835a9 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:19.408750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:19.409602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:19.410832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 143/292] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 96 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:19.417522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4bfeb-8bd7-48af-9b0e-46e5d4575ecd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.436414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4bfeb-8bd7-48af-9b0e-46e5d4575ecd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.450750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.451368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.451643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.451909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.452572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:19.452880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.454503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.464164 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 150/293] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:19.469687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7dfd8a-89bd-4b4b-abc0-e3abf98835a9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.480312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7dfd8a-89bd-4b4b-abc0-e3abf98835a9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.487623 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21ea9d-e796-4f58-903d-3822e60f141e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.488262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21ea9d-e796-4f58-903d-3822e60f141e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.488536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21ea9d-e796-4f58-903d-3822e60f141e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.488806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21ea9d-e796-4f58-903d-3822e60f141e admin admin] PATH_INFO: `/v3/groups/cb6015789fc64607adeeec6a53f8b2f2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.500600 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21ea9d-e796-4f58-903d-3822e60f141e admin admin] RBAC: Authorizing `identity:get_group(group_id=cb6015789fc64607adeeec6a53f8b2f2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.502640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21ea9d-e796-4f58-903d-3822e60f141e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.504368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 144/294] 10.222.0.25 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/groups/cb6015789fc64607adeeec6a53f8b2f2 => generated 229 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:19.510252 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.522383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee196a-5158-4c17-aeba-e57c350a77ff admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.527705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b70ae-c7e0-4b7c-b19e-a8311afa7861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.528403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b70ae-c7e0-4b7c-b19e-a8311afa7861 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.528675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b70ae-c7e0-4b7c-b19e-a8311afa7861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.528942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b70ae-c7e0-4b7c-b19e-a8311afa7861 admin admin] PATH_INFO: `/v3/projects/e25fb0b05a884f4d989636700db0537f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.530035 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b70ae-c7e0-4b7c-b19e-a8311afa7861 admin admin] RBAC: Authorizing `identity:get_project(project_id=e25fb0b05a884f4d989636700db0537f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.532034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b70ae-c7e0-4b7c-b19e-a8311afa7861 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.533924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 151/295] 10.222.0.25 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/projects/e25fb0b05a884f4d989636700db0537f => generated 344 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:19.541258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21ea9d-e796-4f58-903d-3822e60f141e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.555324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21ea9d-e796-4f58-903d-3822e60f141e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.559834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fffb2-d6a8-4618-8257-ecaed741f2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.560541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fffb2-d6a8-4618-8257-ecaed741f2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.560815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fffb2-d6a8-4618-8257-ecaed741f2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.561137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fffb2-d6a8-4618-8257-ecaed741f2b0 admin admin] PATH_INFO: `/v3/projects/e25fb0b05a884f4d989636700db0537f/groups/cb6015789fc64607adeeec6a53f8b2f2/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.563068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fffb2-d6a8-4618-8257-ecaed741f2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e25fb0b05a884f4d989636700db0537f, group_id=cb6015789fc64607adeeec6a53f8b2f2, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.565971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fffb2-d6a8-4618-8257-ecaed741f2b0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.586295 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 145/296] 10.222.0.25 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:19 2022] PUT /identity/v3/projects/e25fb0b05a884f4d989636700db0537f/groups/cb6015789fc64607adeeec6a53f8b2f2/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:20.245960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef1861-7815-460b-a222-0ee2c58f3484 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.246716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef1861-7815-460b-a222-0ee2c58f3484 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.246716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef1861-7815-460b-a222-0ee2c58f3484 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.247851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 152/297] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:20.268624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30017f9-9ce9-4315-b2eb-958997508ff5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.268935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30017f9-9ce9-4315-b2eb-958997508ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.269155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30017f9-9ce9-4315-b2eb-958997508ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.291451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a30017f9-9ce9-4315-b2eb-958997508ff5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:20.292627 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a30017f9-9ce9-4315-b2eb-958997508ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:20.337749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:20.350162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebd045-982f-4982-835f-a07ec03eb137 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.350461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebd045-982f-4982-835f-a07ec03eb137 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.350635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebd045-982f-4982-835f-a07ec03eb137 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.352299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 153/298] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:20.358471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1766b8-f2c7-443f-961f-c2eaff0f61b3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.358753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1766b8-f2c7-443f-961f-c2eaff0f61b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.359033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1766b8-f2c7-443f-961f-c2eaff0f61b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.382811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 146/299] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:20.384471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8b1766b8-f2c7-443f-961f-c2eaff0f61b3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:20.385680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8b1766b8-f2c7-443f-961f-c2eaff0f61b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:20.497005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 154/300] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:20.638813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961aedd-53dc-49ec-92b1-f72432216d87 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.639154 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961aedd-53dc-49ec-92b1-f72432216d87 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.639485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961aedd-53dc-49ec-92b1-f72432216d87 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.640841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 147/301] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:20.646301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2cc3-a8e8-42d6-86cf-48502800c449 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.646588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2cc3-a8e8-42d6-86cf-48502800c449 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.646862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2cc3-a8e8-42d6-86cf-48502800c449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.670895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2eaf2cc3-a8e8-42d6-86cf-48502800c449 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:20.672077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2eaf2cc3-a8e8-42d6-86cf-48502800c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:20.674191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:20.674658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961aedd-53dc-49ec-92b1-f72432216d87 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:20.789990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961aedd-53dc-49ec-92b1-f72432216d87 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:20.793317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 155/302] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:20.826125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7974f-de11-467a-96d8-4f5ea90a8ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:20.826768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7974f-de11-467a-96d8-4f5ea90a8ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.827044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7974f-de11-467a-96d8-4f5ea90a8ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.827313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7974f-de11-467a-96d8-4f5ea90a8ba9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.838489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7974f-de11-467a-96d8-4f5ea90a8ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d5b7974f-de11-467a-96d8-4f5ea90a8ba9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:20.843501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:20.844358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:20.845527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 148/303] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:20.850564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2cc3-a8e8-42d6-86cf-48502800c449 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:20.965205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2cc3-a8e8-42d6-86cf-48502800c449 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.011626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.012239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.012514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.012783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.013446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:21.013743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.015230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.022869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.025204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7974f-de11-467a-96d8-4f5ea90a8ba9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.043753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 156/304] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:21.059942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.092211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.212811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7974f-de11-467a-96d8-4f5ea90a8ba9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.224532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3659253-92c9-4f16-86df-06ee88ca8a31 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.257603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3204a-6c7b-4b13-8ae2-586dedc9d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.258268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3204a-6c7b-4b13-8ae2-586dedc9d8fc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.258783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3204a-6c7b-4b13-8ae2-586dedc9d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.259140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3204a-6c7b-4b13-8ae2-586dedc9d8fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.265447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb175b-f45b-4226-ac56-77f48993f47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.266052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb175b-f45b-4226-ac56-77f48993f47c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.266356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb175b-f45b-4226-ac56-77f48993f47c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.266625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb175b-f45b-4226-ac56-77f48993f47c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.270964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3204a-6c7b-4b13-8ae2-586dedc9d8fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f0e3204a-6c7b-4b13-8ae2-586dedc9d8fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:21.277672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:21.278584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:21.279740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 149/305] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:21.280384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3204a-6c7b-4b13-8ae2-586dedc9d8fc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.358869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.361812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb175b-f45b-4226-ac56-77f48993f47c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.363604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb175b-f45b-4226-ac56-77f48993f47c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.385292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.397228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3204a-6c7b-4b13-8ae2-586dedc9d8fc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.434294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b549c-ccd3-4e6b-acf1-b9bf341e018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.435021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b549c-ccd3-4e6b-acf1-b9bf341e018c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.435365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b549c-ccd3-4e6b-acf1-b9bf341e018c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.435699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b549c-ccd3-4e6b-acf1-b9bf341e018c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.446699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b549c-ccd3-4e6b-acf1-b9bf341e018c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-713b549c-ccd3-4e6b-acf1-b9bf341e018c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:21.453320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:21.454453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:21.455232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:21.457002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 150/306] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:21.458716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b549c-ccd3-4e6b-acf1-b9bf341e018c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.481721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 157/307] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:21.487213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb175b-f45b-4226-ac56-77f48993f47c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.592512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b549c-ccd3-4e6b-acf1-b9bf341e018c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.596830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.600327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb175b-f45b-4226-ac56-77f48993f47c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.633143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.634205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.634682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.635146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.636007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:21.636464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.637998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.639706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.640557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.640970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.641390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.642204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:21.642636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.644228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.647539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 151/308] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:21.648449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.654388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 158/309] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:21.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.661266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.752569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e905dee-8042-4ee9-8463-409fd8a0aba3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.771288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c75ee8-6748-421a-b039-ec9fb34515b0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.787104 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4639-7949-456f-92d5-167711dcf471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.787762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4639-7949-456f-92d5-167711dcf471 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.788026 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4639-7949-456f-92d5-167711dcf471 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.788294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4639-7949-456f-92d5-167711dcf471 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.790757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f4639-7949-456f-92d5-167711dcf471 None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.792135 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f4639-7949-456f-92d5-167711dcf471 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.794390 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 152/310] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:21.794661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4639-7949-456f-92d5-167711dcf471 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.809439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540561e-2eda-4165-942b-f1cc9d07ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.810588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540561e-2eda-4165-942b-f1cc9d07ba3c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.811430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540561e-2eda-4165-942b-f1cc9d07ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.811836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540561e-2eda-4165-942b-f1cc9d07ba3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.852405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.879802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.908210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4639-7949-456f-92d5-167711dcf471 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.919438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540561e-2eda-4165-942b-f1cc9d07ba3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.921563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540561e-2eda-4165-942b-f1cc9d07ba3c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.956801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bdb3f-8b33-408b-a833-12927ca6baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.957673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bdb3f-8b33-408b-a833-12927ca6baf5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.958366 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bdb3f-8b33-408b-a833-12927ca6baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.959255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bdb3f-8b33-408b-a833-12927ca6baf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.042775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bdb3f-8b33-408b-a833-12927ca6baf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.044485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bdb3f-8b33-408b-a833-12927ca6baf5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.060634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 159/311] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:22.062001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540561e-2eda-4165-942b-f1cc9d07ba3c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.136068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.141134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.166244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 153/312] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:22.167036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bdb3f-8b33-408b-a833-12927ca6baf5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.192076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540561e-2eda-4165-942b-f1cc9d07ba3c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.234361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b2513-6368-4ce2-baa7-a8388f069bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.235397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b2513-6368-4ce2-baa7-a8388f069bea None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.235836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b2513-6368-4ce2-baa7-a8388f069bea None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.236353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b2513-6368-4ce2-baa7-a8388f069bea None admin] PATH_INFO: `/v3/groups/97ee3470da97486fb7229f98f17fc652` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.237640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b2513-6368-4ce2-baa7-a8388f069bea None admin] RBAC: Authorizing `identity:get_group(group_id=97ee3470da97486fb7229f98f17fc652)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.239528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b2513-6368-4ce2-baa7-a8388f069bea None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.241280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 160/313] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/groups/97ee3470da97486fb7229f98f17fc652 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.242708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4564151-243f-4bf5-9889-1eb2f5c6ef2a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.243157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4564151-243f-4bf5-9889-1eb2f5c6ef2a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.243712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4564151-243f-4bf5-9889-1eb2f5c6ef2a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.245420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 161/314] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:22.246345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4564151-243f-4bf5-9889-1eb2f5c6ef2a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.284293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bdb3f-8b33-408b-a833-12927ca6baf5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.327576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accc997-e368-4bf1-9623-6a107de15e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.328266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accc997-e368-4bf1-9623-6a107de15e27 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.328548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accc997-e368-4bf1-9623-6a107de15e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.328821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accc997-e368-4bf1-9623-6a107de15e27 None admin] PATH_INFO: `/v3/projects/61e389290239460b8b1a7846922850f0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.329963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accc997-e368-4bf1-9623-6a107de15e27 None admin] RBAC: Authorizing `identity:get_project(project_id=61e389290239460b8b1a7846922850f0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.332508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accc997-e368-4bf1-9623-6a107de15e27 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.333222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 154/315] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/projects/61e389290239460b8b1a7846922850f0 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.333996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accc997-e368-4bf1-9623-6a107de15e27 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.372726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4564151-243f-4bf5-9889-1eb2f5c6ef2a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.400988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.411298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.418154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb202-0c19-4b6e-a3ba-802eea85f384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.418784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb202-0c19-4b6e-a3ba-802eea85f384 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.419062 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb202-0c19-4b6e-a3ba-802eea85f384 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.419331 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb202-0c19-4b6e-a3ba-802eea85f384 None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.420481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeb202-0c19-4b6e-a3ba-802eea85f384 None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.421929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeb202-0c19-4b6e-a3ba-802eea85f384 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.423593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 162/316] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.426604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a214fb9-f722-4518-9c50-4b427984778a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.426989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a214fb9-f722-4518-9c50-4b427984778a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.427274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a214fb9-f722-4518-9c50-4b427984778a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.446041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3a214fb9-f722-4518-9c50-4b427984778a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:22.447585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3a214fb9-f722-4518-9c50-4b427984778a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:22.485121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accc997-e368-4bf1-9623-6a107de15e27 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.530387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.531009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.531285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.531555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.532280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] RBAC: Adding query filter params (group.id=97ee3470da97486fb7229f98f17fc652, role.id=5a2ee20f691f47539f8b1850459be9ca, scope.project.id=61e389290239460b8b1a7846922850f0) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:22.532744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.534548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.542413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 155/317] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/role_assignments?group.id=97ee3470da97486fb7229f98f17fc652&scope.project.id=61e389290239460b8b1a7846922850f0&role.id=5a2ee20f691f47539f8b1850459be9ca => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.542703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 163/318] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:22.543350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.543445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a214fb9-f722-4518-9c50-4b427984778a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.673157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a214fb9-f722-4518-9c50-4b427984778a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.715384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.726530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f912e14-45eb-47fe-b246-bf922412552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.727324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f912e14-45eb-47fe-b246-bf922412552d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.727733 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f912e14-45eb-47fe-b246-bf922412552d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.728132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f912e14-45eb-47fe-b246-bf922412552d None admin] PATH_INFO: `/v3/groups/97ee3470da97486fb7229f98f17fc652` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.728738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5f79c-7352-4127-a20b-123d09f21b4f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.729494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f912e14-45eb-47fe-b246-bf922412552d None admin] RBAC: Authorizing `identity:get_group(group_id=97ee3470da97486fb7229f98f17fc652)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.731316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f912e14-45eb-47fe-b246-bf922412552d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.734079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 164/319] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/groups/97ee3470da97486fb7229f98f17fc652 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.735744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814294d-e2da-4644-89ff-3200ec2b3758 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.736107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814294d-e2da-4644-89ff-3200ec2b3758 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.736940 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814294d-e2da-4644-89ff-3200ec2b3758 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.737751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 165/320] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:22.744679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814294d-e2da-4644-89ff-3200ec2b3758 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.746267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.784452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417e8b3-b2a6-4086-b073-89678f214bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.785249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e8b3-b2a6-4086-b073-89678f214bfb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.785457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e8b3-b2a6-4086-b073-89678f214bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.785725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e8b3-b2a6-4086-b073-89678f214bfb None admin] PATH_INFO: `/v3/groups/97ee3470da97486fb7229f98f17fc652` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.786870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417e8b3-b2a6-4086-b073-89678f214bfb None admin] RBAC: Authorizing `identity:get_group(group_id=97ee3470da97486fb7229f98f17fc652)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.788837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417e8b3-b2a6-4086-b073-89678f214bfb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.790732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 156/321] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/groups/97ee3470da97486fb7229f98f17fc652 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.795614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035e250-9c8c-4082-80f4-12b9ce8c37e8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.795614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035e250-9c8c-4082-80f4-12b9ce8c37e8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.795614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035e250-9c8c-4082-80f4-12b9ce8c37e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.816913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a035e250-9c8c-4082-80f4-12b9ce8c37e8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:22.818355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a035e250-9c8c-4082-80f4-12b9ce8c37e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:22.898464 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814294d-e2da-4644-89ff-3200ec2b3758 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.908549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 157/322] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:22.912684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035e250-9c8c-4082-80f4-12b9ce8c37e8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.935732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.936600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.936881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.937202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.937929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] RBAC: Adding query filter params (group.id=97ee3470da97486fb7229f98f17fc652, role.id=2e66bd66105e4a1e9dcbda045dc2d057, scope.project.id=4122f7e8210a46338e361b98e5347dad) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:22.938272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.939724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.945980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 166/323] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/role_assignments?group.id=97ee3470da97486fb7229f98f17fc652&scope.project.id=4122f7e8210a46338e361b98e5347dad&role.id=2e66bd66105e4a1e9dcbda045dc2d057 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.947009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.032727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.061107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035e250-9c8c-4082-80f4-12b9ce8c37e8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.074688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.090676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd8c1-c7c8-46ab-ada3-8f061baa7617 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.105948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.106737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.107134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.107494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.108614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] RBAC: Adding query filter params (group.id=97ee3470da97486fb7229f98f17fc652, role.id=2e66bd66105e4a1e9dcbda045dc2d057, scope.project.id=61e389290239460b8b1a7846922850f0) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:23.109571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.111506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.118966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 158/324] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/role_assignments?group.id=97ee3470da97486fb7229f98f17fc652&scope.project.id=61e389290239460b8b1a7846922850f0&role.id=2e66bd66105e4a1e9dcbda045dc2d057 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:23.136830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f7d81-7dd8-45de-a5a0-ad22753ffd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.137778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f7d81-7dd8-45de-a5a0-ad22753ffd5a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.138256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f7d81-7dd8-45de-a5a0-ad22753ffd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.138665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f7d81-7dd8-45de-a5a0-ad22753ffd5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.152450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f7d81-7dd8-45de-a5a0-ad22753ffd5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bd3f7d81-7dd8-45de-a5a0-ad22753ffd5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:23.158373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:23.159513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:23.160450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:23.162555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 167/325] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:23.169818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.208167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f7d81-7dd8-45de-a5a0-ad22753ffd5a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.301053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb091c-de96-4065-998d-45b08a16553e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.343838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.344676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.344778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.344996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.346389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f7d81-7dd8-45de-a5a0-ad22753ffd5a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.346513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:23.346513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.349259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.359539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 159/326] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:23.365271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.375478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.390574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec7f09-6c54-438b-bc69-4ee22e75a6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.391723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec7f09-6c54-438b-bc69-4ee22e75a6ed None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.392135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec7f09-6c54-438b-bc69-4ee22e75a6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.392580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec7f09-6c54-438b-bc69-4ee22e75a6ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.407311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec7f09-6c54-438b-bc69-4ee22e75a6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-41ec7f09-6c54-438b-bc69-4ee22e75a6ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:23.412687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:23.413609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:23.414409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 168/327] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:23.418476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.421162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec7f09-6c54-438b-bc69-4ee22e75a6ed None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.495769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351cb1f-471d-40b4-9244-45428f9f9539 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.534614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec7f09-6c54-438b-bc69-4ee22e75a6ed None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.535485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af02c02-2aa5-488d-8487-53862c13dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.536106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af02c02-2aa5-488d-8487-53862c13dd70 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.536385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af02c02-2aa5-488d-8487-53862c13dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.536658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af02c02-2aa5-488d-8487-53862c13dd70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.574850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.575821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.576105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.576380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.577046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:23.577352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.578805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.588919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 169/328] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:23.594298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.617989 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af02c02-2aa5-488d-8487-53862c13dd70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.619919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af02c02-2aa5-488d-8487-53862c13dd70 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.648326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.654903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.717338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42036ce-3d8a-4d79-88d5-5e004ec5418c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.730170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 160/329] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:15:23.735746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af02c02-2aa5-488d-8487-53862c13dd70 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.756460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8107b-cb19-4cea-a047-97842b83c8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.757000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8107b-cb19-4cea-a047-97842b83c8ec None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.757270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8107b-cb19-4cea-a047-97842b83c8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.757534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8107b-cb19-4cea-a047-97842b83c8ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.837529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8107b-cb19-4cea-a047-97842b83c8ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.839051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8107b-cb19-4cea-a047-97842b83c8ec None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.853478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af02c02-2aa5-488d-8487-53862c13dd70 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.885155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.886502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.893837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd874364-ca0c-402d-b3e6-f1f56d14dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.894931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd874364-ca0c-402d-b3e6-f1f56d14dd2b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.895210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd874364-ca0c-402d-b3e6-f1f56d14dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.895483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd874364-ca0c-402d-b3e6-f1f56d14dd2b None admin] PATH_INFO: `/v3/projects/4122f7e8210a46338e361b98e5347dad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.896638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd874364-ca0c-402d-b3e6-f1f56d14dd2b None admin] RBAC: Authorizing `identity:get_project(project_id=4122f7e8210a46338e361b98e5347dad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.898082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd874364-ca0c-402d-b3e6-f1f56d14dd2b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.899877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 161/330] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/projects/4122f7e8210a46338e361b98e5347dad => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:23.905431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd874364-ca0c-402d-b3e6-f1f56d14dd2b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.947971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 170/331] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:15:23.953738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8107b-cb19-4cea-a047-97842b83c8ec None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.002952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd874364-ca0c-402d-b3e6-f1f56d14dd2b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.036226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc95c2-7ff1-475a-bbeb-827fa32b8a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.036784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc95c2-7ff1-475a-bbeb-827fa32b8a49 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.037061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc95c2-7ff1-475a-bbeb-827fa32b8a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.037332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc95c2-7ff1-475a-bbeb-827fa32b8a49 None admin] PATH_INFO: `/v3/groups/97ee3470da97486fb7229f98f17fc652` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.038463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc95c2-7ff1-475a-bbeb-827fa32b8a49 None admin] RBAC: Authorizing `identity:get_group(group_id=97ee3470da97486fb7229f98f17fc652)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.039956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc95c2-7ff1-475a-bbeb-827fa32b8a49 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.041576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 162/332] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/groups/97ee3470da97486fb7229f98f17fc652 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:24.047559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc95c2-7ff1-475a-bbeb-827fa32b8a49 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.049494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8107b-cb19-4cea-a047-97842b83c8ec None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ba702-09a2-4678-93c8-384504eefd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.083215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ba702-09a2-4678-93c8-384504eefd8f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.083487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ba702-09a2-4678-93c8-384504eefd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.083756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ba702-09a2-4678-93c8-384504eefd8f None admin] PATH_INFO: `/v3/projects/e25fb0b05a884f4d989636700db0537f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.084858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ba702-09a2-4678-93c8-384504eefd8f None admin] RBAC: Authorizing `identity:get_project(project_id=e25fb0b05a884f4d989636700db0537f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.086331 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ba702-09a2-4678-93c8-384504eefd8f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.088351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 171/333] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/projects/e25fb0b05a884f4d989636700db0537f => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:24.093716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ba702-09a2-4678-93c8-384504eefd8f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.105701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:24.115965 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:24.149462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc95c2-7ff1-475a-bbeb-827fa32b8a49 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.183304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.183928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.184204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.184465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.185172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] RBAC: Adding query filter params (group.id=97ee3470da97486fb7229f98f17fc652, role.id=5a2ee20f691f47539f8b1850459be9ca, scope.project.id=4122f7e8210a46338e361b98e5347dad) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:24.185478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.187018 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.192490 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 163/334] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:24 2022] GET /identity/v3/role_assignments?group.id=97ee3470da97486fb7229f98f17fc652&scope.project.id=4122f7e8210a46338e361b98e5347dad&role.id=5a2ee20f691f47539f8b1850459be9ca => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:24.194699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ba702-09a2-4678-93c8-384504eefd8f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.231787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7873161-8406-43d1-be7c-a7d2b83859c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.232431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7873161-8406-43d1-be7c-a7d2b83859c4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.232712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7873161-8406-43d1-be7c-a7d2b83859c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.232982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7873161-8406-43d1-be7c-a7d2b83859c4 None admin] PATH_INFO: `/v3/groups/cb6015789fc64607adeeec6a53f8b2f2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.234172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7873161-8406-43d1-be7c-a7d2b83859c4 None admin] RBAC: Authorizing `identity:get_group(group_id=cb6015789fc64607adeeec6a53f8b2f2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.235630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7873161-8406-43d1-be7c-a7d2b83859c4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.237202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 172/335] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:24 2022] GET /identity/v3/groups/cb6015789fc64607adeeec6a53f8b2f2 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:24.243348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.353164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68f58f-f515-48ad-9c0f-73b1b66a6209 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.383084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:24.391392 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f2bf9-8e34-4af6-a9b0-f01fe51e61a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.391998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f2bf9-8e34-4af6-a9b0-f01fe51e61a8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.392267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f2bf9-8e34-4af6-a9b0-f01fe51e61a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.392534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f2bf9-8e34-4af6-a9b0-f01fe51e61a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.393258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f2bf9-8e34-4af6-a9b0-f01fe51e61a8 None admin] RBAC: Adding query filter params (group.id=cb6015789fc64607adeeec6a53f8b2f2, role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=e25fb0b05a884f4d989636700db0537f) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:24.393536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f2bf9-8e34-4af6-a9b0-f01fe51e61a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.395005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f2bf9-8e34-4af6-a9b0-f01fe51e61a8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.401113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 164/336] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:24 2022] GET /identity/v3/role_assignments?group.id=cb6015789fc64607adeeec6a53f8b2f2&scope.project.id=e25fb0b05a884f4d989636700db0537f&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:27.175064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675500a-723c-44d5-a209-a491f89baa1c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.175628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675500a-723c-44d5-a209-a491f89baa1c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.175628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675500a-723c-44d5-a209-a491f89baa1c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.177123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 173/337] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:27.187535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eb6b8-2e33-459a-951b-d7c92b141faf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.187984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eb6b8-2e33-459a-951b-d7c92b141faf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.188065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eb6b8-2e33-459a-951b-d7c92b141faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.215122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cb8eb6b8-2e33-459a-951b-d7c92b141faf None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:27.216320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cb8eb6b8-2e33-459a-951b-d7c92b141faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:27.245795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385be74a-b3e6-466f-9bc4-22a8515e9714 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.246082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385be74a-b3e6-466f-9bc4-22a8515e9714 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.246382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385be74a-b3e6-466f-9bc4-22a8515e9714 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.247987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 174/338] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:27.253249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c6bc-393e-4945-88f1-66a444b0d7b3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.253778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c6bc-393e-4945-88f1-66a444b0d7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.254279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105c6bc-393e-4945-88f1-66a444b0d7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.281287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3105c6bc-393e-4945-88f1-66a444b0d7b3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:27.284081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3105c6bc-393e-4945-88f1-66a444b0d7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:27.331893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 165/339] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:27.392469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 175/340] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:27.455391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4d13-08ea-4380-bbd8-bc43dbd93671 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.455992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4d13-08ea-4380-bbd8-bc43dbd93671 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.456466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4d13-08ea-4380-bbd8-bc43dbd93671 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.458312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 166/341] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:27.463782 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed487-0f14-4b72-ac45-db0564ac0aed None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.464299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed487-0f14-4b72-ac45-db0564ac0aed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.464773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed487-0f14-4b72-ac45-db0564ac0aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.487257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d78ed487-0f14-4b72-ac45-db0564ac0aed None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:27.488768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d78ed487-0f14-4b72-ac45-db0564ac0aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:27.494121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.575683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 176/342] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:27.663531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a4d13-08ea-4380-bbd8-bc43dbd93671 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.791973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a4d13-08ea-4380-bbd8-bc43dbd93671 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.815068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ed487-0f14-4b72-ac45-db0564ac0aed None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.827582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.840144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.841114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.841596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.842161 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.853305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.855126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:27.879499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:27.880525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:27.881394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:27.881394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:27.881394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:27.883255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 167/343] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:27.891384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.934778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ed487-0f14-4b72-ac45-db0564ac0aed None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.975480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.976117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.976395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.976665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.985461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.989385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.991100 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:28.008736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:28.009899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:28.010990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:28.010990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:28.010990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:28.012950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 177/344] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:28.012950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.024536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2896b3f-b60d-4165-9cad-f02bfaf82192 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.070562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.071098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.071402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.071742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.072618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:28.073275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.074677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.088282 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 168/345] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:28.089017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.127483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.132381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ef74-66ec-4bec-8cbf-342457e2f305 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.167812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175258bd-d1c1-42fd-8403-a156ca58c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.168435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175258bd-d1c1-42fd-8403-a156ca58c34e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.169020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175258bd-d1c1-42fd-8403-a156ca58c34e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.169297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175258bd-d1c1-42fd-8403-a156ca58c34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.179996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175258bd-d1c1-42fd-8403-a156ca58c34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.181504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175258bd-d1c1-42fd-8403-a156ca58c34e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.190842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46de8a-45fb-4b0e-ad99-14e30c134cdd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-175258bd-d1c1-42fd-8403-a156ca58c34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:28.196370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:28.197236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:28.198044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:28.198044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:28.198044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:28.198044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 178/346] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:28.203649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935b5ab-bd33-41fc-af48-81239bf5c1f7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.204088 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935b5ab-bd33-41fc-af48-81239bf5c1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.204394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935b5ab-bd33-41fc-af48-81239bf5c1f7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.205667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 179/347] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:28.206398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935b5ab-bd33-41fc-af48-81239bf5c1f7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.233037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.233911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.234374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.234651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.235293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:28.235598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.236964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.253423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 169/348] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:28.253423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.254793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.317621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935b5ab-bd33-41fc-af48-81239bf5c1f7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.353028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462798f-4011-4d28-975c-7233b28e5a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.353577 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462798f-4011-4d28-975c-7233b28e5a9a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.353856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462798f-4011-4d28-975c-7233b28e5a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.354161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462798f-4011-4d28-975c-7233b28e5a9a None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.355223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462798f-4011-4d28-975c-7233b28e5a9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.356670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462798f-4011-4d28-975c-7233b28e5a9a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.363290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.367131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e348767-e492-49d6-9ff9-99aa550fe23f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.378508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 180/349] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:28 2022] POST /identity/v3/users => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:28.380216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373425-829f-4495-80ea-023b5fa23e31 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.380640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373425-829f-4495-80ea-023b5fa23e31 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.381045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373425-829f-4495-80ea-023b5fa23e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.405667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.406329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.406612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.406887 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.407543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:28.407850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.408191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3e373425-829f-4495-80ea-023b5fa23e31 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:28.409339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.409632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3e373425-829f-4495-80ea-023b5fa23e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:28.423044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 170/350] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:28.423725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.484803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 181/351] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:28.486183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3406f9-aa62-41e4-ba21-03c7424ffb4d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.486518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3406f9-aa62-41e4-ba21-03c7424ffb4d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.486793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3406f9-aa62-41e4-ba21-03c7424ffb4d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.488284 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 182/352] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:28.488894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3406f9-aa62-41e4-ba21-03c7424ffb4d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.533382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.554673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1beaf-92eb-4491-b6e9-5e97b20e133c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.597108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824a55a-0259-4aa8-bdc2-d658513499c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.597712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a55a-0259-4aa8-bdc2-d658513499c0 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.598001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a55a-0259-4aa8-bdc2-d658513499c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.598338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a55a-0259-4aa8-bdc2-d658513499c0 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.598990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824a55a-0259-4aa8-bdc2-d658513499c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.600469 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824a55a-0259-4aa8-bdc2-d658513499c0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.624313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 171/353] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:28 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:28.627121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f3a61-47cf-46b2-8745-949145e95f59 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.627409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f3a61-47cf-46b2-8745-949145e95f59 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.627702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f3a61-47cf-46b2-8745-949145e95f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.632305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3406f9-aa62-41e4-ba21-03c7424ffb4d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.651721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5b5f3a61-47cf-46b2-8745-949145e95f59 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:28.653190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5b5f3a61-47cf-46b2-8745-949145e95f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:28.668566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.680441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20196830-4a6b-482a-82e3-7c68a0df9221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.681795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20196830-4a6b-482a-82e3-7c68a0df9221 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.683777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20196830-4a6b-482a-82e3-7c68a0df9221 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.684328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20196830-4a6b-482a-82e3-7c68a0df9221 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.685373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20196830-4a6b-482a-82e3-7c68a0df9221 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.687635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20196830-4a6b-482a-82e3-7c68a0df9221 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.721932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 183/354] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:28 2022] POST /identity/v3/users => generated 248 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:28.773201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20196830-4a6b-482a-82e3-7c68a0df9221 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.777943 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 172/355] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:28.904043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.930191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20196830-4a6b-482a-82e3-7c68a0df9221 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.968616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.969245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.969523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.969794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.980425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.981903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:28.997354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:28.998338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:28.999149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:28.999149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:28.999149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:28.999149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 184/356] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:29.004787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f3a61-47cf-46b2-8745-949145e95f59 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.134898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.160065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f3a61-47cf-46b2-8745-949145e95f59 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.198301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.199616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.200953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.201405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.202323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:29.202795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.204391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.223127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 173/357] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:29.228591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.233081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.343776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e027f4-6b91-45ac-9b76-c789e6e7f10c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.369085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffedb4-36e3-4b94-bbb2-344ed0c14466 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.386997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38411ec2-5ea8-4a48-80bf-748c99b47b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.388013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38411ec2-5ea8-4a48-80bf-748c99b47b0c None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.388501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38411ec2-5ea8-4a48-80bf-748c99b47b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.388972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38411ec2-5ea8-4a48-80bf-748c99b47b0c None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.389816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38411ec2-5ea8-4a48-80bf-748c99b47b0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.391668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38411ec2-5ea8-4a48-80bf-748c99b47b0c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.422346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 174/358] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:29 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:29.444164 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.445185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.445675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.446176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.447205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.448968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.455285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.461139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:15:29.461139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 07:15:29.461139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [parameters: {'id': '853b66ba94dd44259cc25ae6a7c8918c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 07:15:29.461139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:29.482868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return ref.to_dict() Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:29.483824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:15:29.484735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:15:29.485605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:29.487218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:29.487997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'id': '853b66ba94dd44259cc25ae6a7c8918c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:29.488757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 07:15:29.491188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:29.495106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 185/359] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Nov 2 07:15:29 2022] POST /identity/v3/roles => generated 144 bytes in 361 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:29.502192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38411ec2-5ea8-4a48-80bf-748c99b47b0c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.614080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.668839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38411ec2-5ea8-4a48-80bf-748c99b47b0c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.704086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8144092-bcb3-4028-b1ce-b3fdfc68c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.704644 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8144092-bcb3-4028-b1ce-b3fdfc68c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.704929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8144092-bcb3-4028-b1ce-b3fdfc68c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.705200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8144092-bcb3-4028-b1ce-b3fdfc68c6f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.717605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8144092-bcb3-4028-b1ce-b3fdfc68c6f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a8144092-bcb3-4028-b1ce-b3fdfc68c6f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:29.723008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:29.723883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:29.724648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 175/360] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:29.730946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.844459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bbdf9-2ca9-4829-9917-0a171046f4aa None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.880852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e31d-df3a-4864-88ae-ba772d2e34c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.881824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e31d-df3a-4864-88ae-ba772d2e34c6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.882343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e31d-df3a-4864-88ae-ba772d2e34c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.882816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e31d-df3a-4864-88ae-ba772d2e34c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.883682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e31d-df3a-4864-88ae-ba772d2e34c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:29.884582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e31d-df3a-4864-88ae-ba772d2e34c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.886177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e31d-df3a-4864-88ae-ba772d2e34c6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.901013 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 186/361] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:31.124569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5db3-aad1-4aa7-a22f-8629047cab16 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.125110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5db3-aad1-4aa7-a22f-8629047cab16 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.125110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5db3-aad1-4aa7-a22f-8629047cab16 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.126750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 176/362] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:31.132058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359626-f087-48da-b616-5f8e0d6c059c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.132905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359626-f087-48da-b616-5f8e0d6c059c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.133370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359626-f087-48da-b616-5f8e0d6c059c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.155884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7d5ce-b930-406d-aac4-47646f6c2002 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.156210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7d5ce-b930-406d-aac4-47646f6c2002 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.156423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7d5ce-b930-406d-aac4-47646f6c2002 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.156618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ea359626-f087-48da-b616-5f8e0d6c059c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:31.157983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 177/363] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:31.158213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ea359626-f087-48da-b616-5f8e0d6c059c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:31.162759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0ba0-46a4-4bb4-bb0a-b5792035a23b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.163034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0ba0-46a4-4bb4-bb0a-b5792035a23b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.163305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0ba0-46a4-4bb4-bb0a-b5792035a23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.211171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6bcf0ba0-46a4-4bb4-bb0a-b5792035a23b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:31.212812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6bcf0ba0-46a4-4bb4-bb0a-b5792035a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:31.226248 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.242072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 187/364] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:31.347618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 178/365] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:31.387249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a89dd-ceaa-43be-b64d-9f4ab7020546 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.387567 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a89dd-ceaa-43be-b64d-9f4ab7020546 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.387791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a89dd-ceaa-43be-b64d-9f4ab7020546 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.389400 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 188/366] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:31.394598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626b35f-5dfc-44d1-912a-1dc65395e570 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.394881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626b35f-5dfc-44d1-912a-1dc65395e570 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.395150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626b35f-5dfc-44d1-912a-1dc65395e570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.413269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6626b35f-5dfc-44d1-912a-1dc65395e570 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:31.414469 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6626b35f-5dfc-44d1-912a-1dc65395e570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:31.480326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.511386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 179/367] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:31.541568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a89dd-ceaa-43be-b64d-9f4ab7020546 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.647730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a89dd-ceaa-43be-b64d-9f4ab7020546 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.665297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626b35f-5dfc-44d1-912a-1dc65395e570 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.684855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa5533-9e62-4eae-97fe-b832b4af3c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.685487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa5533-9e62-4eae-97fe-b832b4af3c22 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.685761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa5533-9e62-4eae-97fe-b832b4af3c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.686028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa5533-9e62-4eae-97fe-b832b4af3c22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.696979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa5533-9e62-4eae-97fe-b832b4af3c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-56aa5533-9e62-4eae-97fe-b832b4af3c22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:31.702588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:31.703449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:31.704222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 189/368] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:31.708960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa5533-9e62-4eae-97fe-b832b4af3c22 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.766147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.772200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626b35f-5dfc-44d1-912a-1dc65395e570 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.807820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93cc1a-347e-4d5b-a2ba-7f4dc4588fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.808589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93cc1a-347e-4d5b-a2ba-7f4dc4588fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.808957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93cc1a-347e-4d5b-a2ba-7f4dc4588fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.809307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93cc1a-347e-4d5b-a2ba-7f4dc4588fe1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.817320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa5533-9e62-4eae-97fe-b832b4af3c22 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.820579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93cc1a-347e-4d5b-a2ba-7f4dc4588fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7a93cc1a-347e-4d5b-a2ba-7f4dc4588fe1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:31.825076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:31.825960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:31.826758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 180/369] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:31.832172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93cc1a-347e-4d5b-a2ba-7f4dc4588fe1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.876647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.879265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.879848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.880201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.880629 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.881515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:31.881890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:31.883412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:31.915227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 190/370] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:31.920658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.942641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93cc1a-347e-4d5b-a2ba-7f4dc4588fe1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.980333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.981280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.981774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.982333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.983190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:31.983678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:31.985244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:31.994691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 181/371] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:32.000782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.091607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bc7eb-aca4-4f8f-b3d6-619ed9a7e65b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.106401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.109699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235c53d-9d0d-4574-848d-eb43469f8b9a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.111643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.133406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.134131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.134382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.134694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.145653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.147129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.154521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.156243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.156978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.157446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:32.162153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:32.163064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:32.163886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:32.163886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:32.163886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:32.168174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 191/372] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:32.169098 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.172347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.174145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:32.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:32.191832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:32.192661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:32.192661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:32.192661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:32.199575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 182/373] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:32.199575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.298909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647d2a8-ce72-486d-90d2-30eefc325ccf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.315473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee24a6-efe4-4621-a08f-5eb6cd2d5b15 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.332918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9a538-591e-4cc4-93b2-faaee295d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.333605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9a538-591e-4cc4-93b2-faaee295d80b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.333829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9a538-591e-4cc4-93b2-faaee295d80b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.334104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9a538-591e-4cc4-93b2-faaee295d80b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.344960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9a538-591e-4cc4-93b2-faaee295d80b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-71d9a538-591e-4cc4-93b2-faaee295d80b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:32.350628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:32.351525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:32.352340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:32.356612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 192/374] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:32.357511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9a538-591e-4cc4-93b2-faaee295d80b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.358634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.359468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.359931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.360699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.362557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:32.362861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.364246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.385327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 183/375] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:32.386718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.394743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.407248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.486348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9a538-591e-4cc4-93b2-faaee295d80b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.520442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.521060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.521326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.521595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.522100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92998beb-d9e8-4839-a03a-015e81659f06 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.522275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:32.522583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.524046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.541390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 193/376] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:32.541390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.567293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80627fb7-723f-4ed1-a0cd-78fc691bd465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.568150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627fb7-723f-4ed1-a0cd-78fc691bd465 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.568567 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627fb7-723f-4ed1-a0cd-78fc691bd465 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.569017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627fb7-723f-4ed1-a0cd-78fc691bd465 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.569814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80627fb7-723f-4ed1-a0cd-78fc691bd465 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:32.570281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80627fb7-723f-4ed1-a0cd-78fc691bd465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.572461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80627fb7-723f-4ed1-a0cd-78fc691bd465 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.581149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 184/377] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:32.582534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc93ed-f988-411a-8edb-7004f1027491 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.582955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc93ed-f988-411a-8edb-7004f1027491 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.583348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc93ed-f988-411a-8edb-7004f1027491 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.584695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 185/378] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:32.585559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc93ed-f988-411a-8edb-7004f1027491 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.640874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.658009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2230b93-c611-4e03-99b4-afa9173ab34e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.669082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.693103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc93ed-f988-411a-8edb-7004f1027491 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.693944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.694595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.694871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.695139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.702350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.703799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:32.717544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:32.718582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 194/379] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:32.719829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.731855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b69ea8-e425-4929-bfd8-ad3e49038641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.732802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b69ea8-e425-4929-bfd8-ad3e49038641 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.733083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b69ea8-e425-4929-bfd8-ad3e49038641 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.733356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b69ea8-e425-4929-bfd8-ad3e49038641 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.740812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b69ea8-e425-4929-bfd8-ad3e49038641 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.742290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b69ea8-e425-4929-bfd8-ad3e49038641 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-37b69ea8-e425-4929-bfd8-ad3e49038641 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:32.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:32.755892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 186/380] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea9944-b45e-4933-9d06-a43fa79200d5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.762150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea9944-b45e-4933-9d06-a43fa79200d5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.762798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea9944-b45e-4933-9d06-a43fa79200d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.776322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-96ea9944-b45e-4933-9d06-a43fa79200d5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:32.777502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-96ea9944-b45e-4933-9d06-a43fa79200d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:32.817753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cea049-8d41-4556-a047-56cf8a5b421f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.845545 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 187/381] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:32.846803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2e09e-e6b0-4c5c-9479-17b89bbdb544 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.847092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2e09e-e6b0-4c5c-9479-17b89bbdb544 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.847361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2e09e-e6b0-4c5c-9479-17b89bbdb544 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.848726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 188/382] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:32.849253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2e09e-e6b0-4c5c-9479-17b89bbdb544 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.861364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.862442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.862728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.863004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.874190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.875666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.882054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:32.891254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:32.892295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:32.893314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:32.893314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:32.893314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:32.895267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 195/383] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:32.898559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.936549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.978318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2e09e-e6b0-4c5c-9479-17b89bbdb544 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.002431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e16341-aebf-47b6-9377-c315d709d062 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.013935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58974cc1-25ae-4d5f-a9a6-41cc9340e9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.014616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974cc1-25ae-4d5f-a9a6-41cc9340e9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.014929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974cc1-25ae-4d5f-a9a6-41cc9340e9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.015239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974cc1-25ae-4d5f-a9a6-41cc9340e9b9 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.015997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58974cc1-25ae-4d5f-a9a6-41cc9340e9b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:33.016339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58974cc1-25ae-4d5f-a9a6-41cc9340e9b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.017798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58974cc1-25ae-4d5f-a9a6-41cc9340e9b9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.036966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.037563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.037833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.038123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.038855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:33.039133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.040787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.059536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 189/384] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:33.059536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776258c-8639-4eff-807d-a6979e04515c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.059536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776258c-8639-4eff-807d-a6979e04515c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.059536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776258c-8639-4eff-807d-a6979e04515c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.075262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1776258c-8639-4eff-807d-a6979e04515c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:33.076464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1776258c-8639-4eff-807d-a6979e04515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:33.088196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 196/385] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:33.089098 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.144541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.147312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 190/386] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:33.148419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776258c-8639-4eff-807d-a6979e04515c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.188857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ee8d8-49d6-42ce-b2e3-ff7fe940c277 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.210482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.230838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.231462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.231734 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.231998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.232631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:33.232923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.234831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.252220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 197/387] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:33.252220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.263229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776258c-8639-4eff-807d-a6979e04515c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.303809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.304554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.304832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.305108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.315292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.316747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:33.333320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:33.334277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:33.335110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:33.335110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:33.335110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:33.339766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 191/388] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:33.340614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.362574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03d566-8bcd-4b5c-b8da-8d2cc6575902 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.378886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.405918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.406513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.406795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.407061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.418929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.420411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:33.434893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:33.435861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:33.436691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:33.436691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:33.436691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:33.436691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 198/389] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:33.436906 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.449556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137feccc-2fac-48ec-abdf-a01cd1227250 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.457081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.485232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c478b29-f07d-4cfa-8975-17371502bcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.486245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478b29-f07d-4cfa-8975-17371502bcdb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.486679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478b29-f07d-4cfa-8975-17371502bcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.486963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478b29-f07d-4cfa-8975-17371502bcdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.497131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c478b29-f07d-4cfa-8975-17371502bcdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4c478b29-f07d-4cfa-8975-17371502bcdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:33.502279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:33.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:33.504127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 192/390] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:33.504776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c478b29-f07d-4cfa-8975-17371502bcdb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.542518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb067c31-ff2b-4ea2-9859-b0ab71a2a7b3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.575591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.576209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.576485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.576754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.584069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.585528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:33.596738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:33.597594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:33.598520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 199/391] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:33.599069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.611548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c478b29-f07d-4cfa-8975-17371502bcdb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.626778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.647432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.648036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.648306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.648571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.649207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:33.649662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.650958 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.663016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 193/392] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:33.663945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.691630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.702328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a5d05-37e3-4475-bf2a-41855885680a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.737774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.738425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.738755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.739024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.749629 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.751096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.764138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a52f2-aff6-4ea7-b34b-bbf7216d91e0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:33.765221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:33.766197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:33.767097 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:33.767097 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:33.767097 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:33.767097 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 200/393] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:33.767350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.801840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.802470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.802746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.803012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.803637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:33.803939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.805305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.816555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 194/394] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:33.817316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.852866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.873490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552b754-584c-44fa-a64f-8fe06fe1e092 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.902336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.906648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.907326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.907613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.907885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.908581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:33.908892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.910394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.921384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5549ca6-4298-4fcc-bde1-bff61a0a37c0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.922385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 201/395] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:33.923273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.960591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.961239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.961511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.962223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.963203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:33.963498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.964938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.981805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 195/396] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:33.982656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.022983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f697dc-7e42-41eb-a185-cde060a51ccf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.058500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.059118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.059396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.059664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.066031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.067516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.078215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279f639-80d4-456e-9623-92fb87db252d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:34.081481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:34.082461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:34.083280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 202/397] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:34.083744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.090257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:34.114718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.115325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.115601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.115872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.116501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:34.116803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.118269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.129466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 196/398] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:34.130548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.134044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:34.189657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67060a-933e-4a9a-8fbf-d3b859fd5f77 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.223844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.224483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.224755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.225019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.229225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b507cea-c5c5-4ce7-b99d-8bcc74d6bbba None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.231644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.233116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:34.244053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:34.244922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:34.245763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 203/399] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:34.246260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.266959 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.267575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.267849 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.268131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.278137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.279571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:34.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:34.293950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:34.294831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:34.294831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:34.294831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:34.294831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 197/400] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:34.295038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.309604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:34.350073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b48329-5ddd-436e-aea9-5eb39e6828ee None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.355307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:34.383562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.384419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.384708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.384989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.395531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.397008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.399534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca930cb0-b0bb-4220-b240-1c653ee9f1b0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:34.411392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:34.412276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:34.413122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:34.413122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:34.413122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:34.413122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 204/401] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:34.413326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.437369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.437966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.438272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.438551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.439222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:34.439519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.440985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.458167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 198/402] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:34.459136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.516235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928244-76b6-4b56-abe1-fd43b52a539e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.529290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:34.549788 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b927cb6-5435-466a-9ce1-b9125fa27e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.550434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927cb6-5435-466a-9ce1-b9125fa27e0a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.550716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927cb6-5435-466a-9ce1-b9125fa27e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.550985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927cb6-5435-466a-9ce1-b9125fa27e0a None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.551625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b927cb6-5435-466a-9ce1-b9125fa27e0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.553096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b927cb6-5435-466a-9ce1-b9125fa27e0a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.562277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b5304-d53d-4b0c-9cc9-072adfb2307b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.575992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 205/403] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:34 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:34.576818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b927cb6-5435-466a-9ce1-b9125fa27e0a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.579391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:34.600723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.601664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.602021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.602307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.603130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:34.603474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.605006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.620445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 199/404] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:34.621282 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.692089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b927cb6-5435-466a-9ce1-b9125fa27e0a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.728505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.729121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.729396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.729662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.730343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:34.730641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.732054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.740874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19ce79-5654-4b3a-9b05-48cc981db02d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.744732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 206/405] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:34.745572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.781028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.781833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.781968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.782277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.782975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:34.783133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:34.783283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.784748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.799509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 200/406] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:34.800329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.841035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:34.858974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61909-5331-434a-9cb4-7c182cba34dc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.901387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99596da3-8464-4663-99cc-60a62b3b5fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.902211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99596da3-8464-4663-99cc-60a62b3b5fb3 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.902526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99596da3-8464-4663-99cc-60a62b3b5fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.902856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99596da3-8464-4663-99cc-60a62b3b5fb3 None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/0722fc844689413796c658582ce00eb9/roles/636139c60bd74dcaa328bf316f518d4e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.909036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c1117-0c8d-4b80-aa01-bb95c3f780a2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:34.919543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99596da3-8464-4663-99cc-60a62b3b5fb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=0722fc844689413796c658582ce00eb9, role_id=636139c60bd74dcaa328bf316f518d4e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:34.920734 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99596da3-8464-4663-99cc-60a62b3b5fb3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:34.935087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 207/407] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:34 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/0722fc844689413796c658582ce00eb9/roles/636139c60bd74dcaa328bf316f518d4e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:34.935924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99596da3-8464-4663-99cc-60a62b3b5fb3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:34.979441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38849a-10c4-4181-8014-468f01e6b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:34.980031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38849a-10c4-4181-8014-468f01e6b3e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:34.980306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38849a-10c4-4181-8014-468f01e6b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:34.980578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38849a-10c4-4181-8014-468f01e6b3e3 None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/f9fd09151e284f639c93699b0d26db05/roles/636139c60bd74dcaa328bf316f518d4e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:34.998671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38849a-10c4-4181-8014-468f01e6b3e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=f9fd09151e284f639c93699b0d26db05, role_id=636139c60bd74dcaa328bf316f518d4e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.000133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38849a-10c4-4181-8014-468f01e6b3e3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.018754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 201/408] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:34 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/f9fd09151e284f639c93699b0d26db05/roles/636139c60bd74dcaa328bf316f518d4e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:35.019499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38849a-10c4-4181-8014-468f01e6b3e3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:35.069185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:35.138526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99596da3-8464-4663-99cc-60a62b3b5fb3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:35.142462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:35.154927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38849a-10c4-4181-8014-468f01e6b3e3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:35.194065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:35.195525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.196108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.197628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.201393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:35.202247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.202793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.203070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.208411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.210001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.216462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.218251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:35.231452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:35.232546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:35.233599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 208/409] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:35.234353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:35.235445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:35.237670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 202/410] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:35.241241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:35.241390 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:35.372338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bafcd-9a08-4541-90fc-b3637babe6e8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:35.392104 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6fd1c-b915-4f6e-9c88-de51db68e022 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:35.419541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:35.420540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.421010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.422279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.423226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:35.425608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.427361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.444750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:35.455573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:35.456188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.456474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.456751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.457450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:35.457764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.459338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.465703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 209/411] 10.222.0.25 () {56 vars in 1139 bytes} [Wed Nov 2 07:15:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:35.472642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:35.487384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 203/412] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:35.490996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:35.518471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:35.649051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b55e9f-4607-473a-bf3a-1d67a391ffaf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:35.673062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd92c07-00f0-48de-9e69-bb7718126344 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:35.710828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:35.711460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.712109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.712381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.726719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.728262 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.733475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4cd45-7c5e-47a5-b803-a8621ed88709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:35.734157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4cd45-7c5e-47a5-b803-a8621ed88709 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.734439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4cd45-7c5e-47a5-b803-a8621ed88709 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.734712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4cd45-7c5e-47a5-b803-a8621ed88709 None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/d17ba458c5004646b5e21cabcb54fd20/roles/636139c60bd74dcaa328bf316f518d4e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:35.758018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:35.759245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:35.760094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:35.760094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:35.760094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:35.760094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 210/413] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:35.762762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4cd45-7c5e-47a5-b803-a8621ed88709 None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=d17ba458c5004646b5e21cabcb54fd20, role_id=636139c60bd74dcaa328bf316f518d4e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.764451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4cd45-7c5e-47a5-b803-a8621ed88709 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.766852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:35.792511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 204/414] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:35 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/d17ba458c5004646b5e21cabcb54fd20/roles/636139c60bd74dcaa328bf316f518d4e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:35.824274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:35.882346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6523d-ff08-4d00-9432-fa6b7c54f101 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:35.920048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:35.920667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.920947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.921219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.921879 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:35.922298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.923745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.941903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 211/415] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:35.947531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4cd45-7c5e-47a5-b803-a8621ed88709 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.022255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.082777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4cd45-7c5e-47a5-b803-a8621ed88709 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.120330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.120956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.121228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.121491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.128288 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.129732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:36.141955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:36.142834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:36.143660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 205/416] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:36.148893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.277056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b11276-a7b5-4aa8-8fb3-4de7530e91cf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.290178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.317526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2895346-04b1-42ed-bfcf-ac75aa57a696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.318648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2895346-04b1-42ed-bfcf-ac75aa57a696 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.319127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2895346-04b1-42ed-bfcf-ac75aa57a696 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.319656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2895346-04b1-42ed-bfcf-ac75aa57a696 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.320609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2895346-04b1-42ed-bfcf-ac75aa57a696 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:36.321150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2895346-04b1-42ed-bfcf-ac75aa57a696 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.322925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2895346-04b1-42ed-bfcf-ac75aa57a696 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.343257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 212/417] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:36.349800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.485202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96db3-f07c-46ff-9fb0-3434fa10bcc7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.503559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.529659 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b221648-b8d8-40d0-af96-eeb3b420296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.530254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221648-b8d8-40d0-af96-eeb3b420296c None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.530530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221648-b8d8-40d0-af96-eeb3b420296c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.530795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221648-b8d8-40d0-af96-eeb3b420296c None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/c4ad0e5567bd452eadd14ed6ab1edc21/roles/636139c60bd74dcaa328bf316f518d4e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.557066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b221648-b8d8-40d0-af96-eeb3b420296c None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=c4ad0e5567bd452eadd14ed6ab1edc21, role_id=636139c60bd74dcaa328bf316f518d4e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.558582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b221648-b8d8-40d0-af96-eeb3b420296c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.581223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 206/418] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:36 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/c4ad0e5567bd452eadd14ed6ab1edc21/roles/636139c60bd74dcaa328bf316f518d4e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:36.934795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e2010-fe3c-4ea6-9f64-9e6999c42af8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.935121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e2010-fe3c-4ea6-9f64-9e6999c42af8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.935338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e2010-fe3c-4ea6-9f64-9e6999c42af8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.936973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 213/419] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:36.942421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62239fa-3476-4520-b5ae-bfc7dd780782 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.942701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62239fa-3476-4520-b5ae-bfc7dd780782 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.942974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62239fa-3476-4520-b5ae-bfc7dd780782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.962495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f62239fa-3476-4520-b5ae-bfc7dd780782 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:36.963683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f62239fa-3476-4520-b5ae-bfc7dd780782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:37.048744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 207/420] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:37.451141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e2010-fe3c-4ea6-9f64-9e6999c42af8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.567860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227657fa-8cac-4b76-9216-71d5b21fe163 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.568388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227657fa-8cac-4b76-9216-71d5b21fe163 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.568388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227657fa-8cac-4b76-9216-71d5b21fe163 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.570009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 208/421] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:37.574834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3b565-d1c1-437c-a8ae-4b43ca25351a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.575166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3b565-d1c1-437c-a8ae-4b43ca25351a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.575431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3b565-d1c1-437c-a8ae-4b43ca25351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.599358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-23c3b565-d1c1-437c-a8ae-4b43ca25351a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:37.600520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-23c3b565-d1c1-437c-a8ae-4b43ca25351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:37.616037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e2010-fe3c-4ea6-9f64-9e6999c42af8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:37.657378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9faea-c560-4566-bca2-78d3f9b9e308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:37.658001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9faea-c560-4566-bca2-78d3f9b9e308 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.658308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9faea-c560-4566-bca2-78d3f9b9e308 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.658581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9faea-c560-4566-bca2-78d3f9b9e308 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.668894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9faea-c560-4566-bca2-78d3f9b9e308 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5ff9faea-c560-4566-bca2-78d3f9b9e308 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:37.673814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:37.674752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:37.675607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:37.678265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 214/422] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:37.681405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9faea-c560-4566-bca2-78d3f9b9e308 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.716704 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:37.752555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 209/423] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:37.807281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9faea-c560-4566-bca2-78d3f9b9e308 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:37.863741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:37.864347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.864618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.864883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.865539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:37.865831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:37.867234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:37.880678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 215/424] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:37.886564 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3b565-d1c1-437c-a8ae-4b43ca25351a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.928482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.006573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3b565-d1c1-437c-a8ae-4b43ca25351a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.040758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.041381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.041654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.041920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.052225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.053678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:38.069442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:38.070394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:38.071344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:38.071344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:38.071344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:38.074215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 210/425] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:38.076816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.168171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.182883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.192898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453fa84-9134-43e4-9cc0-a8dbb888f986 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.232249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.232680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.233222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.233560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.234610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:38.234720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.236238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.249612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 216/426] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:38.261316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.332355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.367099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba026d-174f-4417-8f50-2e1df3942ee6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.380982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920e64-eaf1-4f30-83e0-d3014d10939e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.410351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9363c9d-517b-4333-8841-ead310464297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.410987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9363c9d-517b-4333-8841-ead310464297 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.411267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9363c9d-517b-4333-8841-ead310464297 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.411540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9363c9d-517b-4333-8841-ead310464297 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.420039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32dcce-bbe7-4ada-8b48-7950095a2a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.420662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32dcce-bbe7-4ada-8b48-7950095a2a45 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.420935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32dcce-bbe7-4ada-8b48-7950095a2a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.421201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32dcce-bbe7-4ada-8b48-7950095a2a45 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.424639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9363c9d-517b-4333-8841-ead310464297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:38.428754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32dcce-bbe7-4ada-8b48-7950095a2a45 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.430224 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32dcce-bbe7-4ada-8b48-7950095a2a45 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e9363c9d-517b-4333-8841-ead310464297 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:38.431181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:38.432190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:38.433113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:38.435515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 211/427] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:38.436940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ca508-620f-4ce8-87de-d1947e5ba900 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.437226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ca508-620f-4ce8-87de-d1947e5ba900 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.437500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ca508-620f-4ce8-87de-d1947e5ba900 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.442226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 212/428] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:38.443426 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8c69-3fef-4dd1-abe4-0df67b9a0e14 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.443704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8c69-3fef-4dd1-abe4-0df67b9a0e14 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.443973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8c69-3fef-4dd1-abe4-0df67b9a0e14 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5b32dcce-bbe7-4ada-8b48-7950095a2a45 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:38.444335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:38.445173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:38.446162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:38.446706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 213/429] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:38.446706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade8c69-3fef-4dd1-abe4-0df67b9a0e14 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.446967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 217/430] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:38.447595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0577a6-4b49-4ce3-aa16-d02f72a06311 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.447790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0577a6-4b49-4ce3-aa16-d02f72a06311 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.448059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0577a6-4b49-4ce3-aa16-d02f72a06311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.466306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bd0577a6-4b49-4ce3-aa16-d02f72a06311 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:38.467469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bd0577a6-4b49-4ce3-aa16-d02f72a06311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:38.474770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.542261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 218/431] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:38.542261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea509b67-e811-4dac-bc45-cec7622c414b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.542261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea509b67-e811-4dac-bc45-cec7622c414b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.542261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea509b67-e811-4dac-bc45-cec7622c414b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.554813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ea509b67-e811-4dac-bc45-cec7622c414b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:38.556459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ea509b67-e811-4dac-bc45-cec7622c414b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:38.569575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade8c69-3fef-4dd1-abe4-0df67b9a0e14 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.608860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.609611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.609974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.610336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.611115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:38.611478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.612978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.618963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.627556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 214/432] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:38.628509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.645329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 219/433] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:38.646275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea509b67-e811-4dac-bc45-cec7622c414b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.727756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.763604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea509b67-e811-4dac-bc45-cec7622c414b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.802249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.803172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.803444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.803712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.815643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.817116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.834084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943aeac-decf-4aee-ab0b-67a67e322429 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:38.834678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:38.835522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:38.836337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:38.836337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:38.836337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:38.836337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 220/434] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:38.837319 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.874391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.875438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.876173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.876857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.877784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:38.878285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.879883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.899133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 215/435] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:38.900337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.957562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d411c0-0ad4-4cab-81a8-15f543cf543e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.971977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.987679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.001178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0442c-175e-4953-bc9c-99ed49719e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.001810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0442c-175e-4953-bc9c-99ed49719e1e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.002084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0442c-175e-4953-bc9c-99ed49719e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.002379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0442c-175e-4953-bc9c-99ed49719e1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.013439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0442c-175e-4953-bc9c-99ed49719e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-02d0442c-175e-4953-bc9c-99ed49719e1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:39.018659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:39.019544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:39.020324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:39.020941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc038-fe9b-4d1a-a1ec-d673cfff4114 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.021012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 221/436] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:39.021012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0442c-175e-4953-bc9c-99ed49719e1e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.060003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de772-8861-4649-970e-63e037713c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.060530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de772-8861-4649-970e-63e037713c6a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.060803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de772-8861-4649-970e-63e037713c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.061071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de772-8861-4649-970e-63e037713c6a None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.061706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de772-8861-4649-970e-63e037713c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:39.062003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de772-8861-4649-970e-63e037713c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.063409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de772-8861-4649-970e-63e037713c6a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.076239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 216/437] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:39.077049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de772-8861-4649-970e-63e037713c6a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.137896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0442c-175e-4953-bc9c-99ed49719e1e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.180862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.181489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.181781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.182083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.185964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de772-8861-4649-970e-63e037713c6a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.193843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.195346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.204556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:39.211020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:39.212652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:39.213485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:39.213485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:39.213485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:39.213485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 222/438] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:39.214345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.237225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37330b-e7a2-4bc8-8863-69a95563943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.237858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37330b-e7a2-4bc8-8863-69a95563943d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.238174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37330b-e7a2-4bc8-8863-69a95563943d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.238450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37330b-e7a2-4bc8-8863-69a95563943d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.248820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37330b-e7a2-4bc8-8863-69a95563943d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:39.252642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6b37330b-e7a2-4bc8-8863-69a95563943d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:39.253985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:39.254894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:39.255709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 217/439] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:39.256364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37330b-e7a2-4bc8-8863-69a95563943d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.320571 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b35f-78a7-4409-b758-94505d303d08 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.358015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.358665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.358939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.359210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.359867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:39.360167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.361550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.365756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37330b-e7a2-4bc8-8863-69a95563943d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.369312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 223/440] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:39.370116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.400122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ad06a-cdb6-410c-98f0-806632b1e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.400752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ad06a-cdb6-410c-98f0-806632b1e91d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.401034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ad06a-cdb6-410c-98f0-806632b1e91d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.401306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ad06a-cdb6-410c-98f0-806632b1e91d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.408662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ad06a-cdb6-410c-98f0-806632b1e91d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.410162 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ad06a-cdb6-410c-98f0-806632b1e91d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5c2ad06a-cdb6-410c-98f0-806632b1e91d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:39.421148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:39.422220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 218/441] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:39.423314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcd696-36af-408a-8b2f-09c0171f509a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.424054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcd696-36af-408a-8b2f-09c0171f509a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.424054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcd696-36af-408a-8b2f-09c0171f509a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.425252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 219/442] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:39.425767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcd696-36af-408a-8b2f-09c0171f509a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.467201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.473494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.478143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9d28a-b48f-4e5b-b2f7-865f6b52908a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.512509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.513113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.513393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.513656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.514325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:39.514625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.516011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.528297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 224/443] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:39.528508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcd696-36af-408a-8b2f-09c0171f509a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.529094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.562226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.562835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.563111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.563413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.564072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:39.564376 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.565741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.573785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 220/444] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:39.574541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.634659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a7130-2aa9-4f17-a970-570426737ff1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.667675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097a701-8049-48a1-be00-eccc8372b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.668299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097a701-8049-48a1-be00-eccc8372b416 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.668576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097a701-8049-48a1-be00-eccc8372b416 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.668850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097a701-8049-48a1-be00-eccc8372b416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.669144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.672118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205d11-cbfe-4b41-8c2e-5c74559f2bbb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.679043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097a701-8049-48a1-be00-eccc8372b416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.680651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097a701-8049-48a1-be00-eccc8372b416 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9097a701-8049-48a1-be00-eccc8372b416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:39.694896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:39.695805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:39.696726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:39.696726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:39.696726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:39.696726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 225/445] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:39.697185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119a38-5050-44b6-a83c-20e7d58bdcd3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.697462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119a38-5050-44b6-a83c-20e7d58bdcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.697729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119a38-5050-44b6-a83c-20e7d58bdcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.712879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.713507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.713793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.714062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.714835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:39.715137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.715828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-24119a38-5050-44b6-a83c-20e7d58bdcd3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:39.716617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.717065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-24119a38-5050-44b6-a83c-20e7d58bdcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:39.720258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.733990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 221/446] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:39.735423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.782173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 226/447] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:39.783011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24119a38-5050-44b6-a83c-20e7d58bdcd3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.835448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23d628-4742-4d17-bc2d-ad7121ad2dca None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.872285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.873214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.873650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.874044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.881111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.881229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24119a38-5050-44b6-a83c-20e7d58bdcd3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.882917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:39.894902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:39.895839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:39.896819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:39.896819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:39.896819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:39.896819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:39.896819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:39.896819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:39.896819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:39.896819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:39.898680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 222/448] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:39.899776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.920865 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.926949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.927644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.927926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.928198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.938028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.941190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.943416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:39.960288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:39.961218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:39.962705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:39.962705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:39.962705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:39.962705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 227/449] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:39.962912 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.013373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d3d95-d2bd-48b8-ada9-126605ace569 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.048151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.048903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.049280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.049650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.050428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:40.050858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.052323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.063500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 223/450] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:40.064293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.065806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ccecf-5c81-421f-98c7-f4a8622e2096 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.098959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b1a04-8c58-4d58-a703-f350e5ba76ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.099485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1a04-8c58-4d58-a703-f350e5ba76ba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.099762 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1a04-8c58-4d58-a703-f350e5ba76ba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.100031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1a04-8c58-4d58-a703-f350e5ba76ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.142397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.157624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.164094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1e66c-877a-401d-950d-2ea07c036a34 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.169346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b1a04-8c58-4d58-a703-f350e5ba76ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.170877 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b1a04-8c58-4d58-a703-f350e5ba76ba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.198379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.198981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.199255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.199521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.200203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:40.200502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.201911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.214863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 224/451] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:40.215740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.270945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 228/452] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:40.271783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b1a04-8c58-4d58-a703-f350e5ba76ba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.314632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015b812-2faf-4666-b155-9d7bb060f38a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.353674 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.354223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.354500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.354764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.355397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:40.355693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.357068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.363751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.365616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.374638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 225/453] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:40.374638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.375312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b1a04-8c58-4d58-a703-f350e5ba76ba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.410535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f556-bbe7-4428-8ace-e5b17819fe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.411172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f556-bbe7-4428-8ace-e5b17819fe20 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.411449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f556-bbe7-4428-8ace-e5b17819fe20 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.411717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f556-bbe7-4428-8ace-e5b17819fe20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.421970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817f556-bbe7-4428-8ace-e5b17819fe20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8817f556-bbe7-4428-8ace-e5b17819fe20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:40.426692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.427547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 229/454] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.428579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f556-bbe7-4428-8ace-e5b17819fe20 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.475838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5d0ad-3360-452e-95a6-9d7982996a27 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.509386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.510040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.510339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.510602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.518204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.519644 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.527883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f556-bbe7-4428-8ace-e5b17819fe20 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:40.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.531416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 226/455] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.532393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.565148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e4c5f-c256-4ce3-a759-0cd0993561c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.565798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e4c5f-c256-4ce3-a759-0cd0993561c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.566074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e4c5f-c256-4ce3-a759-0cd0993561c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.566379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e4c5f-c256-4ce3-a759-0cd0993561c8 None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/c3ab55070be14274857478b5581d2a3d/roles/636139c60bd74dcaa328bf316f518d4e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.580951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e4c5f-c256-4ce3-a759-0cd0993561c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=c3ab55070be14274857478b5581d2a3d, role_id=636139c60bd74dcaa328bf316f518d4e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.582465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e4c5f-c256-4ce3-a759-0cd0993561c8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.590725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.596669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 230/456] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:40 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/c3ab55070be14274857478b5581d2a3d/roles/636139c60bd74dcaa328bf316f518d4e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:40.597469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e4c5f-c256-4ce3-a759-0cd0993561c8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.613542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.639936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba16e3-2f61-4703-b2ff-d01ae107a464 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.682252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.683046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.683369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.683686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.696657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.698153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:40.715292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.716460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:40.717520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:40.717520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:40.717520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:40.718704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 227/457] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.719568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.737286 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e4c5f-c256-4ce3-a759-0cd0993561c8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.776168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.777399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.777815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.778108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.787109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.788559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:40.800417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.801361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:40.802258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:40.806219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 231/458] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.807701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.829505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6eed-7986-41e4-9472-a44a4372f65a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.840708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.867524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.868672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.869057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.869426 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.870259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.870997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:40.871497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.873327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.886476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 228/459] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:40.886476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.918713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a56221-8fcd-46ab-b473-a124adf7598a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.954665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.955344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.955727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.956048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.956827 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:40.957136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.958773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.978900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 232/460] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:40.979847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.994821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08729c58-f907-4bca-98ab-7ebb0ca54f44 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.028380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.028995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.029266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.029511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.030176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.030474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.031824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.043975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 229/461] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.044707 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.078468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.095109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d368f-eb1a-43a3-86c5-a0d2996c324c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.116319 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.132044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.132573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.132852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.133122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.134343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.134654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.136096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.152544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb66340-2516-410c-bbf1-cc90f2cb62f1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.157985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 233/462] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.157985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.192339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.192964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.193238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.193507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.204046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.205491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:41.221210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.222153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:41.222994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:41.222994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.222994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:41.222994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 230/463] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.223225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.260427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171a90f-4dfb-4dc9-bc9d-dd902bf06b46 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.296329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc842c2-6866-4958-b8a3-baa72a02f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.296621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc842c2-6866-4958-b8a3-baa72a02f697 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.296898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc842c2-6866-4958-b8a3-baa72a02f697 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.297167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc842c2-6866-4958-b8a3-baa72a02f697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.317035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.334370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b7199-bd34-4b4d-9d9f-37dfd63ac5d8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.347180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.369650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc842c2-6866-4958-b8a3-baa72a02f697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.371140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc842c2-6866-4958-b8a3-baa72a02f697 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.372054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.372679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.372962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.373234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.380118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.381570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:41.393069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.393882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:41.394751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 231/464] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.395794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.484213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 234/465] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:41.484213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc842c2-6866-4958-b8a3-baa72a02f697 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.504045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b4350-6dc7-4ece-92ce-285ecfefe241 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.539008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624661d6-ec42-4f71-a0a1-374449196d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.539668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624661d6-ec42-4f71-a0a1-374449196d1b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.539983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624661d6-ec42-4f71-a0a1-374449196d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.540359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624661d6-ec42-4f71-a0a1-374449196d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.555736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.561379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.585170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc842c2-6866-4958-b8a3-baa72a02f697 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.620102 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624661d6-ec42-4f71-a0a1-374449196d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.621560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624661d6-ec42-4f71-a0a1-374449196d1b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.622121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.622726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.623004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.623356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.623986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.624309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.625656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.637117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 235/466] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.637918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.734492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 232/467] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:41.739775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624661d6-ec42-4f71-a0a1-374449196d1b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.740137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f706963-7af8-4114-8b55-16d42dfbda31 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.775812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.776448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.776728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.776995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.777812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.778112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.779664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.783822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.789151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.798118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 236/468] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.798118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.844079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624661d6-ec42-4f71-a0a1-374449196d1b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.885405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.886001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.886315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.886617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.893949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88904eb7-79ac-42a2-b8e8-a3cf34dbf546 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.902379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.903847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:41.923110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.924023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:41.924840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:41.924840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.924840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:41.926254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 233/469] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.927516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.940842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.941879 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.942289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.942658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.950502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.951962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:41.963804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.964701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:41.965608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 237/470] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.966173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.024357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.033015 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.040924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97ee2b-3242-496e-9329-c2c38ec958ba None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.067243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad794019-562f-4134-8c58-bb82d7e23012 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.076253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.076989 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.077653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.077653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.090386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.092175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.104031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.104632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.104912 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.105182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.105907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] RBAC: Adding query filter params (role.id=636139c60bd74dcaa328bf316f518d4e, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=0722fc844689413796c658582ce00eb9) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.106243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:42.106554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:42.107389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:42.113338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:42.113338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:42.113338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:42.113524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.113624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 234/471] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:42.113624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.122530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 238/472] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/role_assignments?user.id=0722fc844689413796c658582ce00eb9&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=636139c60bd74dcaa328bf316f518d4e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.123297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.242811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d5f2-9121-461f-af07-b7cf6cfa42a4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.255847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a944f-c7a4-47a1-a2b7-73ea550c4d2a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.285662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.286241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.286559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.286834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.287484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.287742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.287840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.289168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.303094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 235/473] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.304148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.310042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.310913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.311186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.311457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.312191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.312491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.313918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.320056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.339194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 239/474] 10.222.0.25 () {56 vars in 1139 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.342294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.422574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed1e27-211a-45ac-8a1d-779e005ec393 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.445915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72975c-6ab9-460f-b31e-e9114e57c070 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.472503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.473362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.473975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.474782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.475797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.476202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.478949 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.482603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.483255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.483526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.483793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.491506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.493005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.499835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 236/475] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.500742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:42.508677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:42.509694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:42.511380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 240/476] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:42.517361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.563020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.575049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.610474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481924a2-1fab-4208-b831-41cefcc9cac6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.626775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477b9c-6271-4011-a9d2-6058459e52b8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.645469 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c164938-d6f0-42b6-8dc0-5cf6c6f89ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.646251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c164938-d6f0-42b6-8dc0-5cf6c6f89ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.646417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c164938-d6f0-42b6-8dc0-5cf6c6f89ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.646656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c164938-d6f0-42b6-8dc0-5cf6c6f89ba3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.668180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.669226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.669723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.670222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.678158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.679968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:42.692415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:42.693296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:42.694168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:42.694168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:42.694168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:42.694168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:42.694168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:42.694168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:42.694168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:42.694168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:42.696273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 241/477] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:42.697820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.725441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c164938-d6f0-42b6-8dc0-5cf6c6f89ba3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.726917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c164938-d6f0-42b6-8dc0-5cf6c6f89ba3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.794468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.799355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.802535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055443a-ac8a-4a23-896a-0f47870945d9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.837957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bcfca-f470-4cb0-bd6b-ec70f9519e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.838496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bcfca-f470-4cb0-bd6b-ec70f9519e95 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.838768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bcfca-f470-4cb0-bd6b-ec70f9519e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.839033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bcfca-f470-4cb0-bd6b-ec70f9519e95 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.839787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bcfca-f470-4cb0-bd6b-ec70f9519e95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.840092 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bcfca-f470-4cb0-bd6b-ec70f9519e95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.841505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bcfca-f470-4cb0-bd6b-ec70f9519e95 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.842238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 237/478] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:42.843520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c164938-d6f0-42b6-8dc0-5cf6c6f89ba3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.858118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 242/479] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.858118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbd2e-42f8-40c1-b4a9-5f22efa6c05c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.858118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbd2e-42f8-40c1-b4a9-5f22efa6c05c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.858118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbd2e-42f8-40c1-b4a9-5f22efa6c05c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.858118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 243/480] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:42.858118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbbd2e-42f8-40c1-b4a9-5f22efa6c05c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.956354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c164938-d6f0-42b6-8dc0-5cf6c6f89ba3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.962535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbbd2e-42f8-40c1-b4a9-5f22efa6c05c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.992768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.993579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.993984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.994354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.995111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.995502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.997004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.998376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c7091-ed4f-4928-9ae1-a84c4465cff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.998990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7091-ed4f-4928-9ae1-a84c4465cff7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.999277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7091-ed4f-4928-9ae1-a84c4465cff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.999532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7091-ed4f-4928-9ae1-a84c4465cff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.009504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c7091-ed4f-4928-9ae1-a84c4465cff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.011032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c7091-ed4f-4928-9ae1-a84c4465cff7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.014399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 238/481] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.015543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2a9c7091-ed4f-4928-9ae1-a84c4465cff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:43.026081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:43.027181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:43.028231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:43.028231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:43.028231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:43.031391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 244/482] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:43.032857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d517de-43fb-42e1-ab26-1e6a2aa04f70 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.033358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d517de-43fb-42e1-ab26-1e6a2aa04f70 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.033700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d517de-43fb-42e1-ab26-1e6a2aa04f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.049738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.052064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-08d517de-43fb-42e1-ab26-1e6a2aa04f70 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:43.053391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-08d517de-43fb-42e1-ab26-1e6a2aa04f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:43.061287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.124453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 245/483] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:43.125564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d517de-43fb-42e1-ab26-1e6a2aa04f70 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.135165 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2301b2-b265-4a8d-be5d-fd1161d6ff91 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.171384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.171822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.172098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.172362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.173072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] RBAC: Adding query filter params (role.id=636139c60bd74dcaa328bf316f518d4e, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=f9fd09151e284f639c93699b0d26db05) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.173477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.174872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.181658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 239/484] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/role_assignments?user.id=f9fd09151e284f639c93699b0d26db05&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=636139c60bd74dcaa328bf316f518d4e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.182613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.232905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d517de-43fb-42e1-ab26-1e6a2aa04f70 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.269428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.270499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.270967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.271421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.272374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] RBAC: Adding query filter params (role.id=636139c60bd74dcaa328bf316f518d4e, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=d17ba458c5004646b5e21cabcb54fd20) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.272868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.274800 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.286212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 246/485] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/role_assignments?user.id=d17ba458c5004646b5e21cabcb54fd20&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=636139c60bd74dcaa328bf316f518d4e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.294168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.304881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6aa4c-61bf-4672-a4cc-01e86452823d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.352375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.353078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.353452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.353726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.354441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.354745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.356183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.374250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 240/486] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.376044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.391869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.515162 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.518315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade7273-6b1c-4e4b-9e4a-a4869cf35bd8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.559033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.560006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.560466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.560961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.568851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.571211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:43.583568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:43.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:43.585227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:43.585227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:43.585227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:43.585227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:43.585227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:43.585227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:43.585227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:43.585227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:43.586251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 247/487] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:43.593626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.645585 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa103576-19b4-4cd1-8d98-46a146ce4e34 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.670355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.692765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2883f16-9a0f-42c6-b445-3da90b802101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.693405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2883f16-9a0f-42c6-b445-3da90b802101 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.693679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2883f16-9a0f-42c6-b445-3da90b802101 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.694241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2883f16-9a0f-42c6-b445-3da90b802101 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.707872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2883f16-9a0f-42c6-b445-3da90b802101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:43.711431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7334101-20aa-4d71-b43d-2ce15c5f5f21 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c2883f16-9a0f-42c6-b445-3da90b802101 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:43.713587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:43.714482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:43.715251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:43.716373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 241/488] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:43.727081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2883f16-9a0f-42c6-b445-3da90b802101 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.748435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.749092 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.749367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.749635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.750352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.750673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.752073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.758132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.766246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 248/489] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.779608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.852648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2883f16-9a0f-42c6-b445-3da90b802101 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.938335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8c661-7583-44ab-b05f-14bc8ac50757 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.938799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.939422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.939733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.940083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.940860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.941169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.942595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.961281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 242/490] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.967289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.968752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.978001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.979055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.979527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.980055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.981026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] RBAC: Adding query filter params (role.id=636139c60bd74dcaa328bf316f518d4e, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=c4ad0e5567bd452eadd14ed6ab1edc21) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.981527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.983747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.991626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 249/491] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/role_assignments?user.id=c4ad0e5567bd452eadd14ed6ab1edc21&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=636139c60bd74dcaa328bf316f518d4e => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:44.097556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:44.137978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902823b6-16cc-4986-ac99-78ac77b4153b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.185818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.186437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.186713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.186988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.199294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.200792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:44.226810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:44.228293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:44.230030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:44.230030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:44.230030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:44.230030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 243/492] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:44.235235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.357435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80736d50-5a63-4008-9a23-560ca8237509 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.393469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6909a44-18b7-4804-bcc8-46981fc04808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.394154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909a44-18b7-4804-bcc8-46981fc04808 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.394433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909a44-18b7-4804-bcc8-46981fc04808 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.394700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909a44-18b7-4804-bcc8-46981fc04808 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.395338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6909a44-18b7-4804-bcc8-46981fc04808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:44.395636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6909a44-18b7-4804-bcc8-46981fc04808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.397009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6909a44-18b7-4804-bcc8-46981fc04808 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.413863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 250/493] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:44.416402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.554651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea8eb3-2a6e-4ae7-942b-3c9f92724e8e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.594433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:44.594621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248cacd-934a-47f8-a79e-ad9d43c5e3e9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.595512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248cacd-934a-47f8-a79e-ad9d43c5e3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.595982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248cacd-934a-47f8-a79e-ad9d43c5e3e9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.597555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 251/494] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:44.603536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e969a-7fa2-4074-84eb-8ddfe3a1618c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.603929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e969a-7fa2-4074-84eb-8ddfe3a1618c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.604373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e969a-7fa2-4074-84eb-8ddfe3a1618c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.607565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.608402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.608806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.609199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.617372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.619151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:44.639658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:44.640808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:44.641995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:44.643904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-552e969a-7fa2-4074-84eb-8ddfe3a1618c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:44.646837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-552e969a-7fa2-4074-84eb-8ddfe3a1618c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:44.647299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 244/495] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:44.651803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.729433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:44.766809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 252/496] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:44.814559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c829258-9aed-43bc-bd84-92698ae0936e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.848001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.848612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.848960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.849209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.849935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:44.850297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.851806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.869294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 245/497] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:44.874978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e969a-7fa2-4074-84eb-8ddfe3a1618c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.012909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e969a-7fa2-4074-84eb-8ddfe3a1618c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.056328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfe047-b31b-4f1a-bb13-f1636066d8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.056937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe047-b31b-4f1a-bb13-f1636066d8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.057214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe047-b31b-4f1a-bb13-f1636066d8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.057482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe047-b31b-4f1a-bb13-f1636066d8e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.114318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.133283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.164647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfe047-b31b-4f1a-bb13-f1636066d8e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:45.166132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfe047-b31b-4f1a-bb13-f1636066d8e6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:45.206488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.269794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f7d93-847a-4f5d-a30f-897a6cd2eb3a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.332647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee73e-dc6f-4723-b6f1-de3921da9737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.333275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee73e-dc6f-4723-b6f1-de3921da9737 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.333556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee73e-dc6f-4723-b6f1-de3921da9737 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.333828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee73e-dc6f-4723-b6f1-de3921da9737 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.362413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ee73e-dc6f-4723-b6f1-de3921da9737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:45.371846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 253/498] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-af6ee73e-dc6f-4723-b6f1-de3921da9737 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:45.373713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:45.374632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:45.375809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 246/499] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:45.380772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfe047-b31b-4f1a-bb13-f1636066d8e6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.382010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee73e-dc6f-4723-b6f1-de3921da9737 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.542764 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.579751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee73e-dc6f-4723-b6f1-de3921da9737 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.582152 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.583933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfe047-b31b-4f1a-bb13-f1636066d8e6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.631690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.633211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.633975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.634478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.635337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:45.636010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:45.638798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:45.651230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.652532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.652995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.653471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.658642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 254/500] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:45.663804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.678386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:45.680147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:45.706487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:45.707903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:45.708756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:45.708756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:45.708756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:45.719169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 247/501] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:45.735715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.859280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f723815-2abf-4b82-bee6-d1d446d35cd6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.901237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.902254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.902723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.903177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.919930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:45.921638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:45.923854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:45.937973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:45.938981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:45.939923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:45.939923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:45.939923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:45.942032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 255/502] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:45.947686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.954878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922220d-5a27-4652-9e36-3f48ea7d31fc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.992116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb8eb8-335d-49eb-b125-d25590584cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.992765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb8eb8-335d-49eb-b125-d25590584cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.993329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb8eb8-335d-49eb-b125-d25590584cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.993617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb8eb8-335d-49eb-b125-d25590584cf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.994491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb8eb8-335d-49eb-b125-d25590584cf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:45.994783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb8eb8-335d-49eb-b125-d25590584cf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:45.996596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb8eb8-335d-49eb-b125-d25590584cf7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.015387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 248/503] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:46.016859 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca022-ae3d-4169-81cc-e9176eceedd9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.017155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca022-ae3d-4169-81cc-e9176eceedd9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.017431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca022-ae3d-4169-81cc-e9176eceedd9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.019044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 249/504] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:46.019805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fca022-ae3d-4169-81cc-e9176eceedd9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.043845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.083212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013ffa-fe48-4451-88d9-d9bf79dfa7e1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.126634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631056eb-002c-416a-8207-e467dead609f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.127234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631056eb-002c-416a-8207-e467dead609f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.127509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631056eb-002c-416a-8207-e467dead609f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.127769 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631056eb-002c-416a-8207-e467dead609f None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.128401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631056eb-002c-416a-8207-e467dead609f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:46.128697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631056eb-002c-416a-8207-e467dead609f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.130058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631056eb-002c-416a-8207-e467dead609f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.135304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fca022-ae3d-4169-81cc-e9176eceedd9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.142023 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 256/505] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:46.143070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc464-caf3-481b-ba1d-24ae564771ab None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.143350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc464-caf3-481b-ba1d-24ae564771ab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.143618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc464-caf3-481b-ba1d-24ae564771ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.161375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-33fdc464-caf3-481b-ba1d-24ae564771ab None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:46.162503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33fdc464-caf3-481b-ba1d-24ae564771ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:46.172296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e5d1f-c79d-441f-8758-15d5f469a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.172933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e5d1f-c79d-441f-8758-15d5f469a80e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.173212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e5d1f-c79d-441f-8758-15d5f469a80e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.173486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e5d1f-c79d-441f-8758-15d5f469a80e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.181100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e5d1f-c79d-441f-8758-15d5f469a80e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.182587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e5d1f-c79d-441f-8758-15d5f469a80e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9e9e5d1f-c79d-441f-8758-15d5f469a80e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:46.194303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:46.195191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 250/506] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c257eea-5502-4950-9361-1c4afa92268c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.196966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c257eea-5502-4950-9361-1c4afa92268c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.197588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c257eea-5502-4950-9361-1c4afa92268c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.198679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 251/507] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:46.202866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c257eea-5502-4950-9361-1c4afa92268c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.221197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.240831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 257/508] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:46.241475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdc464-caf3-481b-ba1d-24ae564771ab None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.300524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.328031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c257eea-5502-4950-9361-1c4afa92268c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.359367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdc464-caf3-481b-ba1d-24ae564771ab None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.363841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc1b66-adcc-4470-b534-8b14671dfbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.364500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc1b66-adcc-4470-b534-8b14671dfbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.364782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc1b66-adcc-4470-b534-8b14671dfbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.365055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc1b66-adcc-4470-b534-8b14671dfbf2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.373001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc1b66-adcc-4470-b534-8b14671dfbf2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.374495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc1b66-adcc-4470-b534-8b14671dfbf2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-65cc1b66-adcc-4470-b534-8b14671dfbf2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:46.386519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:46.387517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:46.389059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 252/509] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:46.389624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792eeb4-ddd0-4500-8f2d-6cb4291767e9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.389624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792eeb4-ddd0-4500-8f2d-6cb4291767e9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.389751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792eeb4-ddd0-4500-8f2d-6cb4291767e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.412036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.412630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.412840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.413111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.413527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8792eeb4-ddd0-4500-8f2d-6cb4291767e9 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:46.413798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:46.414114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.415175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8792eeb4-ddd0-4500-8f2d-6cb4291767e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:46.416188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.434181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 258/510] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:46.435362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.488809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.514445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 253/511] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:46.514445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792eeb4-ddd0-4500-8f2d-6cb4291767e9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.556067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15414235-b47f-4a37-9634-7f50b1250500 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.593042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.593245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.593635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.593911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.594305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.594980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:46.595212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.596712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.615074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 259/512] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:46.616629 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.633856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792eeb4-ddd0-4500-8f2d-6cb4291767e9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.671563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f0d84-2292-4e76-aafd-85cb03841b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.673196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f0d84-2292-4e76-aafd-85cb03841b1e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.673706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f0d84-2292-4e76-aafd-85cb03841b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.673909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f0d84-2292-4e76-aafd-85cb03841b1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.674635 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f0d84-2292-4e76-aafd-85cb03841b1e None admin] RBAC: Adding query filter params (role.id=636139c60bd74dcaa328bf316f518d4e, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=c3ab55070be14274857478b5581d2a3d) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:46.674944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f0d84-2292-4e76-aafd-85cb03841b1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.676365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f0d84-2292-4e76-aafd-85cb03841b1e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.682630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 254/513] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/role_assignments?user.id=c3ab55070be14274857478b5581d2a3d&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=636139c60bd74dcaa328bf316f518d4e => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:46.684048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa460-73da-4f54-8e1e-e983c05109eb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.684330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa460-73da-4f54-8e1e-e983c05109eb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.684596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa460-73da-4f54-8e1e-e983c05109eb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.689962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 255/514] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:46.689962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aa460-73da-4f54-8e1e-e983c05109eb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.748457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108ea30-cab5-4aea-8ceb-bfd5eced8762 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.751849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.799314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cb75c-031d-4683-aad9-7007e372aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.799969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb75c-031d-4683-aad9-7007e372aa09 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.800242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb75c-031d-4683-aad9-7007e372aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.800506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb75c-031d-4683-aad9-7007e372aa09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.811491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cb75c-031d-4683-aad9-7007e372aa09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.812976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cb75c-031d-4683-aad9-7007e372aa09 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.816885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aa460-73da-4f54-8e1e-e983c05109eb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5c1cb75c-031d-4683-aad9-7007e372aa09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:46.828894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:46.829951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:46.831242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:46.831242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:46.831242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:46.831242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 260/515] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:46.831465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf494-572b-4767-8cc7-5d72c5f8fa76 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.831780 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf494-572b-4767-8cc7-5d72c5f8fa76 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.831888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf494-572b-4767-8cc7-5d72c5f8fa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.851000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-facdf494-572b-4767-8cc7-5d72c5f8fa76 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:46.852268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-facdf494-572b-4767-8cc7-5d72c5f8fa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:46.856311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.863224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.863854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.864133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.864403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.876799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.878281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:46.893877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:46.894996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:46.895964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:46.895964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:46.895964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:46.895964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 256/516] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:46.895964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.934207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 261/517] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:46.934904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdf494-572b-4767-8cc7-5d72c5f8fa76 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.025182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff2b57-2125-4018-8383-7aa83762c952 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.029963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.049033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdf494-572b-4767-8cc7-5d72c5f8fa76 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.063046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.063763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.064106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.064610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.065263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.067414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:47.080297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:47.081536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:47.082598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:47.087973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 257/518] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:47.087973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.096552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.097456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.097877 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.098392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.099297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.099741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.101255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.114262 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 262/519] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.115425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.136781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.195620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23207316-ffb3-4f48-9260-846c51fc37ef None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.229236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321804a-eb37-4667-9edd-5bdd7ef2a679 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.231107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.231717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.231999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.233036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.233684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.233984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.235399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.250111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 258/520] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.250295 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.273351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.274049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.274622 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.274794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.275405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.275705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.277089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.286274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 263/521] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.286274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.298190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.362978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c46f36-a27a-4904-93b7-a64d71e66b82 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.370645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.396876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb1ec2-abb2-4503-89c7-7e638d40e0b8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.398508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9de998-6480-4636-857e-23872c584168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.400217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9de998-6480-4636-857e-23872c584168 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.400551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9de998-6480-4636-857e-23872c584168 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.400828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9de998-6480-4636-857e-23872c584168 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.401510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9de998-6480-4636-857e-23872c584168 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.403267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9de998-6480-4636-857e-23872c584168 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.427617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 259/522] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:47 2022] POST /identity/v3/users => generated 254 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:47.431597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9de998-6480-4636-857e-23872c584168 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.432094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.432908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.433403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.433760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] PATH_INFO: `/v3/services/network` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.434523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.436028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:47.452480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:47.453747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:47.454840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:47.456742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 264/523] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:47.457744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.540036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.576115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559285b-513d-4fc0-b924-f327b5a89e9e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.593599 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9de998-6480-4636-857e-23872c584168 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.625390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.626836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.626836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.627026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.627715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.628005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.629436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.637829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 265/524] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.638910 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.640446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.641103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.641375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.641637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.651053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.652540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:47.671765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:47.673219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:47.674114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:47.674114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:47.674114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:47.674114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:47.674114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:47.674114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:47.674114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:47.674114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:47.676618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 260/525] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:47.683248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.717283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.793894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8a692-be2e-4458-9616-8e6ee95c2198 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.835644 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5679ba-7533-4f14-af8b-a1a6ad1a59f0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.855669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.856583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.856868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.857140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.858193 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] RBAC: Adding query filter params (name=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.858501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.859946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.861938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.865952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 266/526] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.871962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.885510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.886454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.887269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.887728 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.888929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.889691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.891358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.909767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 261/527] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.917049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.016884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.031424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd0d3c-36c4-47e5-9904-0faf51ac5a8e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.049442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ded551-958d-4c26-8373-3863dc75f369 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.092436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f199b-3fe2-40ed-af5a-955bc89963ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.093041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f199b-3fe2-40ed-af5a-955bc89963ab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.093314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f199b-3fe2-40ed-af5a-955bc89963ab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.093580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f199b-3fe2-40ed-af5a-955bc89963ab None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.094238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f199b-3fe2-40ed-af5a-955bc89963ab None admin] RBAC: Adding query filter params (type=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:48.094538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f199b-3fe2-40ed-af5a-955bc89963ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.095901 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f199b-3fe2-40ed-af5a-955bc89963ab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.110661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 267/528] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:48.113595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d41df7-aa7a-4388-b432-a8696f50684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.114293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d41df7-aa7a-4388-b432-a8696f50684c None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.114569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d41df7-aa7a-4388-b432-a8696f50684c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.114834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d41df7-aa7a-4388-b432-a8696f50684c None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/0722fc844689413796c658582ce00eb9/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.119523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d41df7-aa7a-4388-b432-a8696f50684c None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=0722fc844689413796c658582ce00eb9, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.120968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d41df7-aa7a-4388-b432-a8696f50684c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.141138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 262/529] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:47 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/0722fc844689413796c658582ce00eb9/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:48.975880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56154a-55df-4fce-93ca-f379c727241f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.976439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56154a-55df-4fce-93ca-f379c727241f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.976439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56154a-55df-4fce-93ca-f379c727241f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.977929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 268/530] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:48.983419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883bc22-94ee-407b-89ea-00ffbcc554c8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.983833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883bc22-94ee-407b-89ea-00ffbcc554c8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.984015 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883bc22-94ee-407b-89ea-00ffbcc554c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.011097 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5883bc22-94ee-407b-89ea-00ffbcc554c8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:49.012281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5883bc22-94ee-407b-89ea-00ffbcc554c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:49.097443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 263/531] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:49.515946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56154a-55df-4fce-93ca-f379c727241f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.591811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.634051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56154a-55df-4fce-93ca-f379c727241f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.675758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc22017-0b47-43b7-90be-eb4435438700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.676381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc22017-0b47-43b7-90be-eb4435438700 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.676656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc22017-0b47-43b7-90be-eb4435438700 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.676921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc22017-0b47-43b7-90be-eb4435438700 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.692502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc22017-0b47-43b7-90be-eb4435438700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2fc22017-0b47-43b7-90be-eb4435438700 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:49.697256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:49.698183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:49.699572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 269/532] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:49.704232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883bc22-94ee-407b-89ea-00ffbcc554c8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.811713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.851173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883bc22-94ee-407b-89ea-00ffbcc554c8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.887373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.887979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.888258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.888529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.889194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:49.889498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.890915 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.900441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 264/533] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:49.905099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc22017-0b47-43b7-90be-eb4435438700 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.077986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc22017-0b47-43b7-90be-eb4435438700 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.111155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.128190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066606b-a72d-442f-a05f-75eda54d16a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.128826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066606b-a72d-442f-a05f-75eda54d16a6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.129101 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066606b-a72d-442f-a05f-75eda54d16a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.129366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066606b-a72d-442f-a05f-75eda54d16a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.141351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066606b-a72d-442f-a05f-75eda54d16a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.142985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066606b-a72d-442f-a05f-75eda54d16a6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2066606b-a72d-442f-a05f-75eda54d16a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:50.167457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:50.168445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:50.169772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:50.169772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:50.169772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:50.169772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 270/534] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:50.174874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.298517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9df18-8f0d-48b1-b42b-3bd09292ee26 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.298895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9df18-8f0d-48b1-b42b-3bd09292ee26 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.299073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9df18-8f0d-48b1-b42b-3bd09292ee26 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.300666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 271/535] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:50.305816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd4d84-896b-4b7a-a471-e55c055f86b3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.306241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd4d84-896b-4b7a-a471-e55c055f86b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.306523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd4d84-896b-4b7a-a471-e55c055f86b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.328471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3c553-53c0-4968-9c81-211de4bd871f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.332659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2dbd4d84-896b-4b7a-a471-e55c055f86b3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:50.333844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2dbd4d84-896b-4b7a-a471-e55c055f86b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:50.384833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6704e-d5bd-4c1d-bd16-72ed8c64c77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.385815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6704e-d5bd-4c1d-bd16-72ed8c64c77e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.386313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6704e-d5bd-4c1d-bd16-72ed8c64c77e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.386793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6704e-d5bd-4c1d-bd16-72ed8c64c77e None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.387760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6704e-d5bd-4c1d-bd16-72ed8c64c77e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.388123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6704e-d5bd-4c1d-bd16-72ed8c64c77e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.389558 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6704e-d5bd-4c1d-bd16-72ed8c64c77e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.392793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.407439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 265/536] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.407439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9d557-bb1c-4871-ad42-c056d0dea059 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.407439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9d557-bb1c-4871-ad42-c056d0dea059 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.407439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9d557-bb1c-4871-ad42-c056d0dea059 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.410854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 266/537] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:50.411851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9d557-bb1c-4871-ad42-c056d0dea059 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.426146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 272/538] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:50.427739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aae3d-f93a-4633-aaf4-0656c2ff74bd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.428068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aae3d-f93a-4633-aaf4-0656c2ff74bd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.428350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aae3d-f93a-4633-aaf4-0656c2ff74bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.447903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fd6aae3d-f93a-4633-aaf4-0656c2ff74bd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:50.449230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd6aae3d-f93a-4633-aaf4-0656c2ff74bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:50.534149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 273/539] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:50.607819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9d557-bb1c-4871-ad42-c056d0dea059 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.638281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ed09-f2cb-4927-bbf2-393c835dfe7b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.638620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ed09-f2cb-4927-bbf2-393c835dfe7b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.638830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ed09-f2cb-4927-bbf2-393c835dfe7b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.640318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 274/540] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:50.649514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.650164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.650443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.650705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.655505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dec11-b355-4452-a9cd-a4a9f14d6276 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.655787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dec11-b355-4452-a9cd-a4a9f14d6276 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.656057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dec11-b355-4452-a9cd-a4a9f14d6276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.658299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.659784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:50.672054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:50.673003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:50.673948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 267/541] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:50.686928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.691901 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1a0dec11-b355-4452-a9cd-a4a9f14d6276 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:50.693325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1a0dec11-b355-4452-a9cd-a4a9f14d6276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:50.724296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.769714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 275/542] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:50.795826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dec11-b355-4452-a9cd-a4a9f14d6276 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.799777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.906476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1d03c-3613-4c95-8429-724ab05cbe79 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.908803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dec11-b355-4452-a9cd-a4a9f14d6276 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.951003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.952079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.952524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.952937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.953803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.954272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.955526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a57cfb-60e2-455b-8d83-e2a8bac1d210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.956043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.956349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a57cfb-60e2-455b-8d83-e2a8bac1d210 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.956727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a57cfb-60e2-455b-8d83-e2a8bac1d210 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.957095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a57cfb-60e2-455b-8d83-e2a8bac1d210 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.971198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a57cfb-60e2-455b-8d83-e2a8bac1d210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-00a57cfb-60e2-455b-8d83-e2a8bac1d210 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:50.976220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:50.977934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:50.979026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:50.981283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 276/543] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:50.986908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a57cfb-60e2-455b-8d83-e2a8bac1d210 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.987046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 268/544] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:50.988114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.017598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.105286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904cb8-2172-4f54-90e6-73789ff54269 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.106312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a57cfb-60e2-455b-8d83-e2a8bac1d210 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.120000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.144170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.144791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.145072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.145349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.146018 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:51.146363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.147271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14fced-5275-407a-b811-71740be27589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.147738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.147893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14fced-5275-407a-b811-71740be27589 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.148149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14fced-5275-407a-b811-71740be27589 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.148419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14fced-5275-407a-b811-71740be27589 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.149423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14fced-5275-407a-b811-71740be27589 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.151150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14fced-5275-407a-b811-71740be27589 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.161535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 269/545] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:51.161535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.178139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 277/546] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:50 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:51.178139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14fced-5275-407a-b811-71740be27589 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.279796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955bc6a-aa6d-40d0-81f5-4729ee37d650 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.288220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.313029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14fced-5275-407a-b811-71740be27589 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.318314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.318985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.319256 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.319532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.331656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.333222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:51.349795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:51.350904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:51.351796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:51.351796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:51.351796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:51.351796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 270/547] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:51.351796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.357548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bde68-01de-45c8-ba86-b208a5069940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.358212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bde68-01de-45c8-ba86-b208a5069940 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.358491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bde68-01de-45c8-ba86-b208a5069940 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.358755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bde68-01de-45c8-ba86-b208a5069940 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.359421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bde68-01de-45c8-ba86-b208a5069940 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.360813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bde68-01de-45c8-ba86-b208a5069940 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.379220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 278/548] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:51 2022] POST /identity/v3/services => generated 240 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:51.379999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021456-433a-413b-b847-fece92468ea8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.380285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021456-433a-413b-b847-fece92468ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.380557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021456-433a-413b-b847-fece92468ea8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.381740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 279/549] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:51.382460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98021456-433a-413b-b847-fece92468ea8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.397225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.502146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc248d2-b3d4-404e-b4c7-fc9dbd2fae20 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.552700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98021456-433a-413b-b847-fece92468ea8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.553036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e720f-a94a-4f3b-ab17-419c732c3edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.553659 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e720f-a94a-4f3b-ab17-419c732c3edb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.553935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e720f-a94a-4f3b-ab17-419c732c3edb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.554236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e720f-a94a-4f3b-ab17-419c732c3edb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.576865 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e720f-a94a-4f3b-ab17-419c732c3edb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.579684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e720f-a94a-4f3b-ab17-419c732c3edb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.596502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.597108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.597387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.597652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.598326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:51.598624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-015e720f-a94a-4f3b-ab17-419c732c3edb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:51.598898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:51.599782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:51.601010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:51.601010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:51.601010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:51.601181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.601266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 271/550] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:51.601703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a116cc-922f-4da7-bacd-c0608a6f9106 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.601801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a116cc-922f-4da7-bacd-c0608a6f9106 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.602153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a116cc-922f-4da7-bacd-c0608a6f9106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.607917 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.618736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 280/551] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:51.619486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.632053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b8a116cc-922f-4da7-bacd-c0608a6f9106 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:51.634020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b8a116cc-922f-4da7-bacd-c0608a6f9106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:51.718011 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 272/552] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:51.718720 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a116cc-922f-4da7-bacd-c0608a6f9106 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.735363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.761947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b31d69-9407-4b04-88d9-89a63bb8af00 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.835115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.836234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.837328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.837873 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.839051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:51.839702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.842137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.873903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a116cc-922f-4da7-bacd-c0608a6f9106 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.881104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 281/553] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:51.886696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.953610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.955374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.955654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.955922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.971254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.973019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:51.991961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:51.993250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:51.994147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:51.994147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:51.994147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:51.994147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:51.994147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:51.994147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:51.994147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:51.994147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:51.994579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.998834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 273/554] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:52.004637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.047130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9bdb2-5a98-4790-8136-7670f3044dd1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.085109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.085796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.086072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.086841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.104794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.106365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:52.129203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:52.130183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:52.131479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 282/555] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:52.137160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.195773 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e867822-0845-44e2-89b7-ed5a1684e0d0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.198453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.244046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.244925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.245233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.245994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.246981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.247286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.248984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.265678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879b60f-e09c-4bc8-b2ba-ba9f6f3b8aaf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.266195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 274/556] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:52.267042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.285016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.309671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.310222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.310499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.310770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.311721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.311828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.313194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.331897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 283/557] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.332923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.399918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3d401-c8d2-4640-9599-64a3817edde6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.435721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.442865 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be588cc3-6aea-4ea1-b857-ef344706ba20 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.451270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c20b74-3aa5-43bd-9c1c-c3e92be37411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.452068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c20b74-3aa5-43bd-9c1c-c3e92be37411 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.452439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c20b74-3aa5-43bd-9c1c-c3e92be37411 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.452804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c20b74-3aa5-43bd-9c1c-c3e92be37411 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.466748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c20b74-3aa5-43bd-9c1c-c3e92be37411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e5c20b74-3aa5-43bd-9c1c-c3e92be37411 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:52.471964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:52.473044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:52.474048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:52.476002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 275/558] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:52.477179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c20b74-3aa5-43bd-9c1c-c3e92be37411 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.490985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b411de-42a6-4050-aeaf-b5449ed5b93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.492337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b411de-42a6-4050-aeaf-b5449ed5b93c None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.492936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b411de-42a6-4050-aeaf-b5449ed5b93c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.493344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b411de-42a6-4050-aeaf-b5449ed5b93c None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/c3ab55070be14274857478b5581d2a3d/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.496045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b411de-42a6-4050-aeaf-b5449ed5b93c None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=c3ab55070be14274857478b5581d2a3d, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.497879 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b411de-42a6-4050-aeaf-b5449ed5b93c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.517397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 284/559] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:52 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/c3ab55070be14274857478b5581d2a3d/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:52.519480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b411de-42a6-4050-aeaf-b5449ed5b93c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.608131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.615933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c20b74-3aa5-43bd-9c1c-c3e92be37411 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.663426 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.664283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.665367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.665575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.695402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.696958 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:52.715514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:52.716656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:52.717714 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:52.717714 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:52.717714 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:52.720552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 276/560] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:52.726334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.735659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b411de-42a6-4050-aeaf-b5449ed5b93c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.786508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.787227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.788103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.788212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.789001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.790217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.790217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.790217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.803353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 285/561] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.808875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.918131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73735518-2fd3-44b9-b4cc-6ccdcb94ff0f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.921269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.925409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55979259-85cd-401f-af60-9a77d8a17510 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.953320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.954195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.954665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.955068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.955841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.956277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.957779 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.964361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.965068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.965406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.965784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.972662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 277/562] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.978445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.978809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.980044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:52.995672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:52.996651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:52.997490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:52.997490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:52.997490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:52.997490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 286/563] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:53.002482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.121050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef1332-b3c2-4a0e-9163-6c2994864daa None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.160906 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.161609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.162152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.162429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.163494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:53.164055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.166178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.169164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.172436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ecec8-11c9-4ae6-9d29-2c8a1fd89456 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.182998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 287/564] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:53.195215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.212763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.223298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.223934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.224208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.224537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.232932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.234675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:53.258555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:53.259458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:53.260323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 278/565] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:53.267038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.355187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7c8f-eaef-4fed-950e-597ff7333cbc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.412246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a936a-739f-43a9-9cf1-4ed1cd184cb5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.436195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.437173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.437640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.438136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.445674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.447584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:53.459907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:53.460881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:53.462776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:53.465050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 288/566] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:53.470558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.472143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.476895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.483794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.484415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.485401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:53.485913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.487746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.517124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 279/567] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:53.526344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.566224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.613537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.652106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e832c02-13b7-4f07-a69d-b6dca2087f22 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.670295 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e1d6d-cc79-420b-90e1-b6fce675a0a6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.689044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ca288-38e0-4a47-ad90-67b3d11fc45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.690027 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ca288-38e0-4a47-ad90-67b3d11fc45d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.690583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ca288-38e0-4a47-ad90-67b3d11fc45d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.690991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ca288-38e0-4a47-ad90-67b3d11fc45d None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.691852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ca288-38e0-4a47-ad90-67b3d11fc45d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:53.692309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ca288-38e0-4a47-ad90-67b3d11fc45d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.693965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ca288-38e0-4a47-ad90-67b3d11fc45d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.715694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08519ed8-cd04-41d5-8b5e-e2164b8eca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.716373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519ed8-cd04-41d5-8b5e-e2164b8eca65 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.716649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519ed8-cd04-41d5-8b5e-e2164b8eca65 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.716922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519ed8-cd04-41d5-8b5e-e2164b8eca65 None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/55fd5764905e4dfa899a90270181b220/roles/636139c60bd74dcaa328bf316f518d4e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.719895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 289/568] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:53.722304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992a0af-b1e7-4cd1-911c-d42d50b97fc4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.722730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992a0af-b1e7-4cd1-911c-d42d50b97fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.723137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992a0af-b1e7-4cd1-911c-d42d50b97fc4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.729621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 290/569] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:53.729621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992a0af-b1e7-4cd1-911c-d42d50b97fc4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.738504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08519ed8-cd04-41d5-8b5e-e2164b8eca65 None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=55fd5764905e4dfa899a90270181b220, role_id=636139c60bd74dcaa328bf316f518d4e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.740797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08519ed8-cd04-41d5-8b5e-e2164b8eca65 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.757174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 280/570] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:53 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/55fd5764905e4dfa899a90270181b220/roles/636139c60bd74dcaa328bf316f518d4e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:15:53.759092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2838ea9-01d9-40a0-a80d-9d7b4391fbda None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.759557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2838ea9-01d9-40a0-a80d-9d7b4391fbda None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.760007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2838ea9-01d9-40a0-a80d-9d7b4391fbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.784186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a2838ea9-01d9-40a0-a80d-9d7b4391fbda None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:53.785777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a2838ea9-01d9-40a0-a80d-9d7b4391fbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:53.870923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.881680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 281/571] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:53.935581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992a0af-b1e7-4cd1-911c-d42d50b97fc4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.963319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7822-c078-4df8-b9dc-8ac19a6b2c3a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.963834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7822-c078-4df8-b9dc-8ac19a6b2c3a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.964277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7822-c078-4df8-b9dc-8ac19a6b2c3a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.967354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 282/572] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:53.973517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ababcb-0167-4426-9c6b-77d9eeb732cb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.973517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ababcb-0167-4426-9c6b-77d9eeb732cb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.973774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ababcb-0167-4426-9c6b-77d9eeb732cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.982492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.990268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6e85c-c681-488e-ab3c-32405114fac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.990895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6e85c-c681-488e-ab3c-32405114fac9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.991171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6e85c-c681-488e-ab3c-32405114fac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.991442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6e85c-c681-488e-ab3c-32405114fac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.012727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-04ababcb-0167-4426-9c6b-77d9eeb732cb None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:54.015820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04ababcb-0167-4426-9c6b-77d9eeb732cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:54.095522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6e85c-c681-488e-ab3c-32405114fac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.098077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6e85c-c681-488e-ab3c-32405114fac9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.116721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 283/573] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:54.237721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 291/574] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:15:54.243272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ababcb-0167-4426-9c6b-77d9eeb732cb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.311211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6e85c-c681-488e-ab3c-32405114fac9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.362458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.392037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ababcb-0167-4426-9c6b-77d9eeb732cb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.440787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.442293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.442492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6e85c-c681-488e-ab3c-32405114fac9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.442950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.443425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.454779 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.456569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:54.475867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:54.476940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:54.477968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:54.477968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:54.477968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:54.478704 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.479330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.479604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.479872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.480448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.481195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.481637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 284/575] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:54.481827 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.482951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:54.491280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:54.492268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:54.493187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 292/576] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:54.493956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.600142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a9734-d712-411b-8ca3-6797e2388071 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.621591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.634845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bc4cc-3539-4678-baf9-b27586686169 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.640182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.640914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.641254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.641575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] PATH_INFO: `/v3/services/network` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.642252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.643690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:54.654381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:54.655387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:54.656233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 285/577] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:54.657086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.685044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.686481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.687042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.687551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.688595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.689089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.691284 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.708685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 293/578] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.710057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.711001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.774538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2f089-7f90-4da6-850a-f564ff3f4b6b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.809211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.809996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.810457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.810737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.811476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.812347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.813336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.823514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 286/579] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.824538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.849809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3ad18-0c70-43ad-a899-724061221056 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.892633 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.893397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.893774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.894256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.894985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] RBAC: Adding query filter params (name=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.895381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.896826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.902768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 294/580] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.903697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.923470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.951144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b15b69-49b7-47f3-9be6-facf30649a72 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.966964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.001487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.002539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.002983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.003650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.012490 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.014800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.019658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037a6f1-ca6b-4755-9dd3-bf2311678d22 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:55.032967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:55.034051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:55.035215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:55.035215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:55.035215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:55.035215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:55.035215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:55.035215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:55.035215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:55.035215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:55.037764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 287/581] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:55.039061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.059772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.060599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.060923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.061285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.062052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.062478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.063955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.073783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 295/582] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.074819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.159058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b5aa9-dde2-4c00-bd81-9d8ea9229b6c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.159615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.179676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663772-e58b-4133-8ea7-32d9d65bf6ef None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.197988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.198671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.198951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.199222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.199845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] RBAC: Adding query filter params (type=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.200148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.201527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.207560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 288/583] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.208357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.216549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e28e1f-615d-4bf0-8513-370d7be9d29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.217157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e28e1f-615d-4bf0-8513-370d7be9d29b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.217514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e28e1f-615d-4bf0-8513-370d7be9d29b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.218263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e28e1f-615d-4bf0-8513-370d7be9d29b None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.218958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e28e1f-615d-4bf0-8513-370d7be9d29b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.219259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e28e1f-615d-4bf0-8513-370d7be9d29b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.221352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e28e1f-615d-4bf0-8513-370d7be9d29b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.239483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 296/584] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.240809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5125-959c-46a2-851d-8abb94798d0d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.241091 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5125-959c-46a2-851d-8abb94798d0d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.241359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5125-959c-46a2-851d-8abb94798d0d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.242913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 297/585] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:55.243568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5125-959c-46a2-851d-8abb94798d0d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.248175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.331287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943dddb-a1a2-490b-86f0-4652820fc833 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.350976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5125-959c-46a2-851d-8abb94798d0d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.377391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.378321 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.378779 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.379657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.380841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] RBAC: Adding query filter params (interface=public, service_id=94ee7e78cffb4f54b8bf4cbc89937455, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.381330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.383815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.396285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dc3c2-30e4-4941-9778-29d1442a2c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.396982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc3c2-30e4-4941-9778-29d1442a2c05 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.397358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc3c2-30e4-4941-9778-29d1442a2c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.397883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc3c2-30e4-4941-9778-29d1442a2c05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.397985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 289/586] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/endpoints?service_id=94ee7e78cffb4f54b8bf4cbc89937455&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.398619 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.403973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dc3c2-30e4-4941-9778-29d1442a2c05 None admin] RBAC: Adding query filter params (interface=public, service_id=ef084c41bd124cfca1a08fffbb634581, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.404255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dc3c2-30e4-4941-9778-29d1442a2c05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.405675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dc3c2-30e4-4941-9778-29d1442a2c05 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.419350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 298/587] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/endpoints?service_id=ef084c41bd124cfca1a08fffbb634581&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.419350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598fdd8-471d-4328-9a34-8057810adf5b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.419350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598fdd8-471d-4328-9a34-8057810adf5b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.419350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598fdd8-471d-4328-9a34-8057810adf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.423650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.434954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5598fdd8-471d-4328-9a34-8057810adf5b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:55.436124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5598fdd8-471d-4328-9a34-8057810adf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:55.501795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 299/588] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:55.502469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598fdd8-471d-4328-9a34-8057810adf5b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.506584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.513929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f4eb-74f0-4ca0-8012-1c06140cdfa2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.561766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.563311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.563895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.564345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.565875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=0722fc844689413796c658582ce00eb9) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.566598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.568288 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.580758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 290/589] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/role_assignments?user.id=0722fc844689413796c658582ce00eb9&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.581928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.645073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598fdd8-471d-4328-9a34-8057810adf5b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.707937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca917e3a-1697-4833-8a28-892b40976c6e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.735516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6047edf-6b10-4213-907f-c300c104e0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.736226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6047edf-6b10-4213-907f-c300c104e0bd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.736595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6047edf-6b10-4213-907f-c300c104e0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.736956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6047edf-6b10-4213-907f-c300c104e0bd None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.737662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6047edf-6b10-4213-907f-c300c104e0bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.739171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6047edf-6b10-4213-907f-c300c104e0bd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.740669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.753654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb9c7f-b728-4563-b4ee-0f57efe81265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.754556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb9c7f-b728-4563-b4ee-0f57efe81265 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.755158 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb9c7f-b728-4563-b4ee-0f57efe81265 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.755674 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb9c7f-b728-4563-b4ee-0f57efe81265 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.756457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb9c7f-b728-4563-b4ee-0f57efe81265 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.758003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb9c7f-b728-4563-b4ee-0f57efe81265 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.758188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 300/590] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/services => generated 772 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.771191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6047edf-6b10-4213-907f-c300c104e0bd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.772318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 291/591] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.928033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6047edf-6b10-4213-907f-c300c104e0bd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.968015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130784b-bd9f-4443-bd7b-23cc8c32e0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.968667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130784b-bd9f-4443-bd7b-23cc8c32e0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.968941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130784b-bd9f-4443-bd7b-23cc8c32e0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.969207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130784b-bd9f-4443-bd7b-23cc8c32e0f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.982061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130784b-bd9f-4443-bd7b-23cc8c32e0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1130784b-bd9f-4443-bd7b-23cc8c32e0f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:55.987562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:55.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:55.989395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 301/592] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:55.995374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb9c7f-b728-4563-b4ee-0f57efe81265 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.025429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:56.108785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb9c7f-b728-4563-b4ee-0f57efe81265 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.147160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fccd4-d5fa-45ab-afcf-a4899b99d54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.148018 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fccd4-d5fa-45ab-afcf-a4899b99d54e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.148437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fccd4-d5fa-45ab-afcf-a4899b99d54e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.149106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fccd4-d5fa-45ab-afcf-a4899b99d54e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.150033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fccd4-d5fa-45ab-afcf-a4899b99d54e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:56.150505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fccd4-d5fa-45ab-afcf-a4899b99d54e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.152013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fccd4-d5fa-45ab-afcf-a4899b99d54e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.161122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 292/593] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:56.170214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130784b-bd9f-4443-bd7b-23cc8c32e0f8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.242832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d7e3-6777-4b3a-ab2f-9f924d1ed06f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.243194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d7e3-6777-4b3a-ab2f-9f924d1ed06f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.243560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d7e3-6777-4b3a-ab2f-9f924d1ed06f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.245143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 293/594] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:56.250436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9da714-13cb-4995-b1ab-39133564b871 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.250716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9da714-13cb-4995-b1ab-39133564b871 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.250983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9da714-13cb-4995-b1ab-39133564b871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.270508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:56.295495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bb9da714-13cb-4995-b1ab-39133564b871 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:56.296827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bb9da714-13cb-4995-b1ab-39133564b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:56.304246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130784b-bd9f-4443-bd7b-23cc8c32e0f8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.370662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.371725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.372202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.372672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.385813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.387606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.396021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 294/595] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:56.403618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:56.404582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:56.405484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:56.405484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:56.405484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:56.407529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 302/596] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:56.413384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9da714-13cb-4995-b1ab-39133564b871 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.433479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:56.515969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9da714-13cb-4995-b1ab-39133564b871 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.551955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.552924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.553447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.553941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.554858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:56.555551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.557409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.570042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 295/597] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:56.576118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.704105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed70ca-d913-4339-8031-5b171a3c61f8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.716020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:56.740321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.741366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.741865 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.742386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.750915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.752680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:56.764315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:56.765347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:56.766361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:56.768851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 303/598] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:56.772273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.776226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.862498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:56.888089 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a902-c71d-46c0-9aae-0b4e203c1efc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.937480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44076c-6a3e-4a8b-8546-4f010906ca63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.938541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44076c-6a3e-4a8b-8546-4f010906ca63 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.938957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44076c-6a3e-4a8b-8546-4f010906ca63 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.939356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44076c-6a3e-4a8b-8546-4f010906ca63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.950507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44076c-6a3e-4a8b-8546-4f010906ca63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ef44076c-6a3e-4a8b-8546-4f010906ca63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:56.956173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:56.957083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:56.957916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:56.960039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 296/599] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:56.968433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44076c-6a3e-4a8b-8546-4f010906ca63 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.002715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca170d8b-f619-45c3-9ba0-1e04d49f51e4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.072721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.073334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.073614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.073889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.074656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:57.074964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.076389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.107881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44076c-6a3e-4a8b-8546-4f010906ca63 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.117872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 304/600] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:57.123517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.147123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.147735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.148016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.148287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.149049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:57.149353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.150826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.156933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:57.161184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 297/601] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:57.167012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.199107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:57.281099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e463c1c-3e22-404f-a146-9f90a0a264ce None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.320879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.321582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.321861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.322221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.334255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.336355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:57.353275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:57.354304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:57.356286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:57.356286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:57.356286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:57.356286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 298/602] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:57.360842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.379087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36d776-79fc-4266-9214-26b9a19ef87f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.423520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:57.474478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe34419-b918-4d55-ac88-75ae8fb2b8d9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.483409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e49638-f54c-4939-a479-362da6008299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.484033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e49638-f54c-4939-a479-362da6008299 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.484310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e49638-f54c-4939-a479-362da6008299 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.484582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e49638-f54c-4939-a479-362da6008299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.510566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.511800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.512151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.512496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.513214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:57.513609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.515141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.528689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 299/603] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:57.534262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.640330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0f637-73d6-48c2-9a9b-fe5a1286deca None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.666455 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:57.677613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.678523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.678900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.679288 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.687523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.689067 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:57.703286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:57.705563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:57.707869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 300/604] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:57.709370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e49638-f54c-4939-a479-362da6008299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.709370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e49638-f54c-4939-a479-362da6008299 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.714110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.750979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:57.837316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 305/605] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 07:15:57.847122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e49638-f54c-4939-a479-362da6008299 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.891152 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1501e-8567-401a-947d-8a802662cf31 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.938755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.939362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.939636 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.939904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.940631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:57.940928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.942378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.983040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 301/606] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:57.991107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.015021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e49638-f54c-4939-a479-362da6008299 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.055667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.065537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.074748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.075753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.076306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.076786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.088375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.090569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:58.107753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:58.108748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:58.109608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:58.109608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:58.109608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:58.115447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 306/607] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:58.118305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.226432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeecc1a-7d21-4020-a25c-b59318aff3be None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.267385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.268424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.268927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.269395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.270363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:58.270936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.272562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.284161 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb44fb-a611-4698-a817-c47b409131d9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.286828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 307/608] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:58.293676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.375537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc56e6e-6b00-41ed-adea-5a0ecb24a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.376183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56e6e-6b00-41ed-adea-5a0ecb24a917 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.376457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56e6e-6b00-41ed-adea-5a0ecb24a917 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.376724 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56e6e-6b00-41ed-adea-5a0ecb24a917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.395117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.429743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63da74-1579-4db7-817f-cab660994848 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.454722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc56e6e-6b00-41ed-adea-5a0ecb24a917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.456255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc56e6e-6b00-41ed-adea-5a0ecb24a917 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.477011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d56983-2880-4263-9d92-42c3e7285463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.477779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d56983-2880-4263-9d92-42c3e7285463 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.477868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d56983-2880-4263-9d92-42c3e7285463 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.478229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d56983-2880-4263-9d92-42c3e7285463 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.486694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d56983-2880-4263-9d92-42c3e7285463 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.488151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d56983-2880-4263-9d92-42c3e7285463 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.490818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f6d56983-2880-4263-9d92-42c3e7285463 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:58.500675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:58.501586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:58.502473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:58.502473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:58.502473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:58.502473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:58.502473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:58.502473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:58.502473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:58.502473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:58.506724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 308/609] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:58.507851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67096f-9a4b-406a-a7c8-9c3c6da6f241 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.508346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67096f-9a4b-406a-a7c8-9c3c6da6f241 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.509249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67096f-9a4b-406a-a7c8-9c3c6da6f241 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.515096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 309/610] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:58.515096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6d2f8-791b-49a3-ab15-1d8c75875645 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.515096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6d2f8-791b-49a3-ab15-1d8c75875645 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.515096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6d2f8-791b-49a3-ab15-1d8c75875645 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.515096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 310/611] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:58.515096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6d2f8-791b-49a3-ab15-1d8c75875645 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.569711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 302/612] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:15:58.571312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748500-4786-4e91-9ce7-6f06103e47df None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.571666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748500-4786-4e91-9ce7-6f06103e47df None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.571960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748500-4786-4e91-9ce7-6f06103e47df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.590718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ee748500-4786-4e91-9ce7-6f06103e47df None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:58.591919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ee748500-4786-4e91-9ce7-6f06103e47df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:58.613313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.617963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6d2f8-791b-49a3-ab15-1d8c75875645 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.651724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.652240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.652516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.652785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.653470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:58.653770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.655198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.660057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 303/613] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:58.661460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77d34-079f-4868-b0ef-2372c7a1b639 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.661740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77d34-079f-4868-b0ef-2372c7a1b639 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.662014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77d34-079f-4868-b0ef-2372c7a1b639 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.666210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 304/614] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:15:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:58.667303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 311/615] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:58.667794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6301b-16fd-4702-9eba-ff0cb317a5c7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.668259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.668400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6301b-16fd-4702-9eba-ff0cb317a5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.668695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6301b-16fd-4702-9eba-ff0cb317a5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.687600 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-78d6301b-16fd-4702-9eba-ff0cb317a5c7 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:58.689390 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-78d6301b-16fd-4702-9eba-ff0cb317a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:58.747078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.762214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 305/616] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:58.764111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243665f-712a-4722-bf82-624f8d6500d8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.764562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243665f-712a-4722-bf82-624f8d6500d8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.765220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243665f-712a-4722-bf82-624f8d6500d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.788565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae286e-b644-4450-9818-67e97dacb8e9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.788861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a243665f-712a-4722-bf82-624f8d6500d8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:58.790237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a243665f-712a-4722-bf82-624f8d6500d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:58.834307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.835178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.835545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.835912 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.850170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.852173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:58.871874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:58.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:58.873949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:58.873949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:58.873949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:58.874830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 312/617] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:58.875835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.882464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 306/618] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:58.883520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243665f-712a-4722-bf82-624f8d6500d8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.904728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.022393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243665f-712a-4722-bf82-624f8d6500d8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.026832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96885ed4-94e9-415c-94a7-4d8a7e2bd0d5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.044844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.061187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.061994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.062309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.062582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.063151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.063262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.063508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.063747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.064071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.064342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.064877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.065059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=c3ab55070be14274857478b5581d2a3d) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.065364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.066801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.082373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 313/619] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/role_assignments?user.id=c3ab55070be14274857478b5581d2a3d&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.082373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.089320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 307/620] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.090488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.164022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.193171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e1987-7917-4459-b1c5-99e54055edad None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.210466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27c6b-abca-46f2-bcf0-dc5530a9eaf1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.239410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.240050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.240326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.240594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.241182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.242661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.247665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.248306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.248584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.248856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.249445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.250859 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.252313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.253277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:59.254180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 314/621] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.255547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.261103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.263405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:59.265334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:59.267095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 308/622] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.267231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.326525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.368006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca32e7-e2c8-4a78-a5ba-2e0fb1ee8294 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.384658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fc7d2-1b98-43ec-b3e5-4276544d9104 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.402627 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.403242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.403517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.403783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.415541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.417733 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.421813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.425221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.426270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.426546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.426777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] PATH_INFO: `/v3/services/network` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.427370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.428789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:59.432366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.433276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.434215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:59.434215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:59.434215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:59.434215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:59.434215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:59.434215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:59.434215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:59.434215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:15:59.438424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 315/623] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.439718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.439992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.440951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:15:59.441816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 309/624] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.442696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.554115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e0fbf-5c3c-4d2b-839b-461372075cfa None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.554557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d759275-6582-484b-8415-c17d6a31bc58 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.564566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.590577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.591863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.592084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.592199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.592389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.592644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.592922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.593013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.593189 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.593280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.593814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.594131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.596022 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.596140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.600374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 310/625] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.600992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.607299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 316/626] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.607299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.675973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.713512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f56d2-aa85-49bd-a36e-adf5bd2d4df0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.713725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fca20-7ec8-4ae7-bff1-05dcdbf644b7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.748224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.748826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.749131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.749358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.749747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.750037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.750387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.750480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.750659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.750936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.751564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] RBAC: Adding query filter params (name=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.751802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.751893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.753211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.758073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 317/627] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.758914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.768156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 311/628] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.769189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.803805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.870548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8802a-8cb4-4196-be84-e059a485a339 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.875676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a18b8-0c1d-4c49-8e61-91622992c9cf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.900466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.909355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.909964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.910273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.910543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.911174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.911473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.912870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.914274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.915221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.915634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.916059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.916982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.917428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.917966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 318/629] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.918785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.919235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.931309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 312/630] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.931309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.046767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.054435 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59015d1e-7b7d-422d-aa5c-d1016889bbf2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.058921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4519a4-07dc-41ac-ba1d-f178e69595cb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.103556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.104579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.105098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.105620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.106605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.106800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] RBAC: Adding query filter params (role.id=636139c60bd74dcaa328bf316f518d4e, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=55fd5764905e4dfa899a90270181b220) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:00.107264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.107480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.107858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.108224 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.108992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] RBAC: Adding query filter params (type=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:00.109387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.109794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.111001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.117797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 313/631] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/role_assignments?user.id=55fd5764905e4dfa899a90270181b220&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=636139c60bd74dcaa328bf316f518d4e => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:00.119178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.129377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 319/632] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:00.129377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.229930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.283322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1e432-86ca-44ac-9b9d-61ec8e1880c2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.287238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96486530-35df-432b-baf4-25bd6177b550 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.323392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95614d1-3f4f-4fe3-842e-fd415d830ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.324422 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95614d1-3f4f-4fe3-842e-fd415d830ebe None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.324978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95614d1-3f4f-4fe3-842e-fd415d830ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.325455 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95614d1-3f4f-4fe3-842e-fd415d830ebe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.326364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95614d1-3f4f-4fe3-842e-fd415d830ebe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.328186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95614d1-3f4f-4fe3-842e-fd415d830ebe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.331974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c21f3-80c3-49d1-8680-fda859044581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.332602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c21f3-80c3-49d1-8680-fda859044581 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.332902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c21f3-80c3-49d1-8680-fda859044581 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.333133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c21f3-80c3-49d1-8680-fda859044581 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.333805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c21f3-80c3-49d1-8680-fda859044581 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.335331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c21f3-80c3-49d1-8680-fda859044581 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.360959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 320/633] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:00 2022] POST /identity/v3/endpoints => generated 337 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:00.375127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 314/634] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:01.292170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219e5b-1cb4-464b-b9e8-e3c4d6b0dc73 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.292677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219e5b-1cb4-464b-b9e8-e3c4d6b0dc73 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.292677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219e5b-1cb4-464b-b9e8-e3c4d6b0dc73 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.294246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 321/635] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:01.299713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d0729-dae7-4bef-8a44-80d85f4ea75d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.300109 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d0729-dae7-4bef-8a44-80d85f4ea75d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.300273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d0729-dae7-4bef-8a44-80d85f4ea75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.318962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dd1d0729-dae7-4bef-8a44-80d85f4ea75d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:01.320130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dd1d0729-dae7-4bef-8a44-80d85f4ea75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:01.331548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:01.411044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 315/636] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:01.684499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae219e5b-1cb4-464b-b9e8-e3c4d6b0dc73 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.792312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae219e5b-1cb4-464b-b9e8-e3c4d6b0dc73 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.806131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:01.828482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.829106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.829379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.829650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.830372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.831818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:01.841495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:01.842484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:01.843569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 322/637] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:01.848906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d0729-dae7-4bef-8a44-80d85f4ea75d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.964933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d0729-dae7-4bef-8a44-80d85f4ea75d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.999447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d729bf9-2946-471b-9994-ccad9d2694bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:02.000401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d729bf9-2946-471b-9994-ccad9d2694bc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.000869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d729bf9-2946-471b-9994-ccad9d2694bc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.001314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d729bf9-2946-471b-9994-ccad9d2694bc None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.002153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d729bf9-2946-471b-9994-ccad9d2694bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:02.002634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d729bf9-2946-471b-9994-ccad9d2694bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:02.004207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d729bf9-2946-471b-9994-ccad9d2694bc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:02.009640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 316/638] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:02.015182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:02.220325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15369ee2-5c8f-4ea3-8e20-9f4aa6b9b8e3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:02.273084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c70f6a-0932-4ed1-b686-16f1596d7b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:02.274239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c70f6a-0932-4ed1-b686-16f1596d7b03 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.274849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c70f6a-0932-4ed1-b686-16f1596d7b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.275365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c70f6a-0932-4ed1-b686-16f1596d7b03 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.276352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c70f6a-0932-4ed1-b686-16f1596d7b03 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:02.279090 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c70f6a-0932-4ed1-b686-16f1596d7b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:02.280809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c70f6a-0932-4ed1-b686-16f1596d7b03 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:02.287746 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 323/639] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:02.397472 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58bfe-a857-481e-bd85-a8bd4b3b5ecd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.398020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58bfe-a857-481e-bd85-a8bd4b3b5ecd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.398020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58bfe-a857-481e-bd85-a8bd4b3b5ecd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.399675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 317/640] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:02.405328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89871754-8195-41ce-92c8-375cec4c9a5e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.405835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89871754-8195-41ce-92c8-375cec4c9a5e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.405938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89871754-8195-41ce-92c8-375cec4c9a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.447876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-89871754-8195-41ce-92c8-375cec4c9a5e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:02.449276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-89871754-8195-41ce-92c8-375cec4c9a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:02.505448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:02.544621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 324/641] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:02.584695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eceac-444d-4cd5-8c73-ae2edce87ba2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.584695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eceac-444d-4cd5-8c73-ae2edce87ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.584695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eceac-444d-4cd5-8c73-ae2edce87ba2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.585387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 318/642] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:02.591285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a97de-38e1-4d15-99f7-0003b8254e4c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.591748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a97de-38e1-4d15-99f7-0003b8254e4c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.592160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a97de-38e1-4d15-99f7-0003b8254e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.610572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1c5a97de-38e1-4d15-99f7-0003b8254e4c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:02.611945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1c5a97de-38e1-4d15-99f7-0003b8254e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:02.679138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 325/643] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:02.829903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c41c82-a4cf-43d2-be07-393437fd8895 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.829903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c41c82-a4cf-43d2-be07-393437fd8895 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.829903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c41c82-a4cf-43d2-be07-393437fd8895 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.830537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 319/644] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:02.833049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a97de-38e1-4d15-99f7-0003b8254e4c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:02.835874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4e075-96bf-4f48-8905-5db19db29418 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.836729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4e075-96bf-4f48-8905-5db19db29418 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.837233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4e075-96bf-4f48-8905-5db19db29418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.851217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:02.858253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-48e4e075-96bf-4f48-8905-5db19db29418 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:02.860055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-48e4e075-96bf-4f48-8905-5db19db29418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:02.938601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:02.939291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 320/645] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:02.941630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a97de-38e1-4d15-99f7-0003b8254e4c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:02.965348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4e075-96bf-4f48-8905-5db19db29418 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:02.977407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddea8f4-213a-4d80-892b-4292537bb5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:02.978160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea8f4-213a-4d80-892b-4292537bb5ca None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.978531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea8f4-213a-4d80-892b-4292537bb5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.979009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea8f4-213a-4d80-892b-4292537bb5ca None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.979881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddea8f4-213a-4d80-892b-4292537bb5ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:02.981425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddea8f4-213a-4d80-892b-4292537bb5ca None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.006117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 326/646] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:02 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:03.081513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4e075-96bf-4f48-8905-5db19db29418 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.122774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.128705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244558bb-fcfc-414e-a301-19e958a1aa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.129355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244558bb-fcfc-414e-a301-19e958a1aa28 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.129672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244558bb-fcfc-414e-a301-19e958a1aa28 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.129942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244558bb-fcfc-414e-a301-19e958a1aa28 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.142458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244558bb-fcfc-414e-a301-19e958a1aa28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-244558bb-fcfc-414e-a301-19e958a1aa28 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:03.147387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:03.148339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:03.149334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 321/647] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:03.155208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddea8f4-213a-4d80-892b-4292537bb5ca None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.214182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244558bb-fcfc-414e-a301-19e958a1aa28 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.284681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddea8f4-213a-4d80-892b-4292537bb5ca None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.321032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.321649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.321927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.322242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.322913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.323217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.324743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.334849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 327/648] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:03.340272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.342971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244558bb-fcfc-414e-a301-19e958a1aa28 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.371215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.384274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.384932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.385214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.385491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.386530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.387790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:03.397415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:03.399225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:03.400058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:03.402168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 322/649] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:03.404748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.466983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.468288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6d706-aa7a-49c1-8ada-72b66283a792 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.503480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.504198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.504475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.504840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.513840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5988e3d-f0dd-44b4-b3e2-084d5858f37f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.519510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.520964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:03.540672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:03.541638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:03.542519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:03.542519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:03.542519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:03.542519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 328/650] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:03.548380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.557106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.557736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.558000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.558296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.558934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.559239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.560609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.571628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 323/651] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:03.573002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.646340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.677766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71714679-6d83-48cb-a793-bf2516881873 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.687647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a704cf-ece5-4ebb-8dea-4fdd11e3320a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.708157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.718563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.719024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.719358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.719632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.720277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.720573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.721966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.724538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.725215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.725550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.725886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.726698 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.727025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.728433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.733277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 324/652] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:03.736397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 329/653] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:03.742023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.857592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b301151-3d4a-430f-ad0b-3e23b95510f6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.895126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.895766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.896040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.896311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.904976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.906372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:03.918218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:03.919123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:03.920050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:03.923988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 325/654] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:03.925173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.030444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c96c1-8257-4967-8810-4334e9dc6c52 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.065638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.066658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.067148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.067676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.068585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:04.069075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.070785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.072670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.089961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 330/655] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:04.095547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.143201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.198056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2430-05a9-4480-b7cd-813f2acb5309 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.232478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.233175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.233377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.233646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.244862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.246352 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:04.261241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:04.262230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:04.263107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:04.263107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:04.263107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.263107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 326/656] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:04.268276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.372660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f34762-6e80-4060-b1ff-76f837881898 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.407896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.408493 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.408770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.409035 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.409673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:04.409969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.411419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.423302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 331/657] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:04.428802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.528294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.534261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9781d9-f9b8-493c-81db-a24436961e20 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.573977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f3271-238c-4b9d-b5da-470ad48f4e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.574665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3271-238c-4b9d-b5da-470ad48f4e2d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.574983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3271-238c-4b9d-b5da-470ad48f4e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.575257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3271-238c-4b9d-b5da-470ad48f4e2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.582775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f3271-238c-4b9d-b5da-470ad48f4e2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.584230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f3271-238c-4b9d-b5da-470ad48f4e2d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-591f3271-238c-4b9d-b5da-470ad48f4e2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:04.596020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:04.596900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:04.597757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 327/658] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:04.603114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.657411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.669287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003624b-91f9-483d-ab89-248ea80269bc None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.669575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003624b-91f9-483d-ab89-248ea80269bc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.669826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003624b-91f9-483d-ab89-248ea80269bc None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.671421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 328/659] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:04.679298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e5d7d-3987-4406-a80a-a0ee192a3931 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.679576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e5d7d-3987-4406-a80a-a0ee192a3931 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.679944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e5d7d-3987-4406-a80a-a0ee192a3931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.699002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-513e5d7d-3987-4406-a80a-a0ee192a3931 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:04.700322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-513e5d7d-3987-4406-a80a-a0ee192a3931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:04.717495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cea83-ff57-4966-88e8-394191f0db7b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.754454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.755514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.756051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.756520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.757574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:04.758239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.759884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.785330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 329/660] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:04 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:04.785517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 332/661] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:04.786818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e5d7d-3987-4406-a80a-a0ee192a3931 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.874563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.886796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e5d7d-3987-4406-a80a-a0ee192a3931 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.921838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd3d38-0e60-48d5-99ec-f8af6a03d57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.922769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd3d38-0e60-48d5-99ec-f8af6a03d57a None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.923204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd3d38-0e60-48d5-99ec-f8af6a03d57a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.923606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd3d38-0e60-48d5-99ec-f8af6a03d57a None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/55fd5764905e4dfa899a90270181b220/roles/f049cc06faaa41d99bb5f9f49c1d1096` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.925972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd3d38-0e60-48d5-99ec-f8af6a03d57a None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=55fd5764905e4dfa899a90270181b220, role_id=f049cc06faaa41d99bb5f9f49c1d1096)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.927654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd3d38-0e60-48d5-99ec-f8af6a03d57a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.944831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 330/662] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:04 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/55fd5764905e4dfa899a90270181b220/roles/f049cc06faaa41d99bb5f9f49c1d1096 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:05.059311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.169952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c9e9c-1571-4e94-b063-03f54116df70 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.189107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.214131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170532d-f61e-4a04-bc4d-44fd11041681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.214766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170532d-f61e-4a04-bc4d-44fd11041681 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.215045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170532d-f61e-4a04-bc4d-44fd11041681 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.215315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170532d-f61e-4a04-bc4d-44fd11041681 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.215992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170532d-f61e-4a04-bc4d-44fd11041681 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.217377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170532d-f61e-4a04-bc4d-44fd11041681 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.238250 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 333/663] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:05 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:05.312747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcccb49-9b27-4c3f-9ee9-4c117e6131bf None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.312929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcccb49-9b27-4c3f-9ee9-4c117e6131bf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.313193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcccb49-9b27-4c3f-9ee9-4c117e6131bf None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.314906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 331/664] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:05.321232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412c04-5ad4-4305-9b83-dc538f5da716 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.321232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412c04-5ad4-4305-9b83-dc538f5da716 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.321232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412c04-5ad4-4305-9b83-dc538f5da716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.342356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-27412c04-5ad4-4305-9b83-dc538f5da716 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:05.343649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-27412c04-5ad4-4305-9b83-dc538f5da716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:05.426650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 334/665] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:05 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:05.699089 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcccb49-9b27-4c3f-9ee9-4c117e6131bf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.799932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcccb49-9b27-4c3f-9ee9-4c117e6131bf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.838787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce601ca-891c-436c-a62a-10e53c76d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.839787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce601ca-891c-436c-a62a-10e53c76d53b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.840277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce601ca-891c-436c-a62a-10e53c76d53b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.840752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce601ca-891c-436c-a62a-10e53c76d53b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.841525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce601ca-891c-436c-a62a-10e53c76d53b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.843117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce601ca-891c-436c-a62a-10e53c76d53b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9ce601ca-891c-436c-a62a-10e53c76d53b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:05.852464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:05.853359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:05.854255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:05.856928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 332/666] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:05.863454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412c04-5ad4-4305-9b83-dc538f5da716 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.974838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.979477 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412c04-5ad4-4305-9b83-dc538f5da716 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.015975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.016584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.016861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.017130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.017760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.018061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.019456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.022926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12d16e-b991-438b-973a-374db1b7f810 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.023210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12d16e-b991-438b-973a-374db1b7f810 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.023552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12d16e-b991-438b-973a-374db1b7f810 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.025196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 333/667] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:06.026336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 335/668] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.029996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f464d42-e1a7-499d-b405-4e0272c317bf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.030302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f464d42-e1a7-499d-b405-4e0272c317bf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.030575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f464d42-e1a7-499d-b405-4e0272c317bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.032132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.049329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4f464d42-e1a7-499d-b405-4e0272c317bf None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:06.050540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4f464d42-e1a7-499d-b405-4e0272c317bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:06.058353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.132694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 334/669] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:06 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:06.147959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1a8de-f586-4b8c-8a54-ffec06392fde None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.183280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.183884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.184161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.184431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.185060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.185360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.186802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.192190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 336/670] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:06.198287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f464d42-e1a7-499d-b405-4e0272c317bf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.302062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f464d42-e1a7-499d-b405-4e0272c317bf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.341406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.342012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.342314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.342586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.343283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] RBAC: Adding query filter params (interface=public, service_id=5335a7544199484db8d9e56c0181655d, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.343593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.344963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.350334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 335/671] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/endpoints?service_id=5335a7544199484db8d9e56c0181655d&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.359466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.388799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.406124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.407964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.480695 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b7089-e328-4691-acd1-22698c05b706 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.507237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2132f4-d4dd-43ba-a861-637ed991da00 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.522586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d945f1-d1c8-486e-8768-d55516f4e3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.524316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d945f1-d1c8-486e-8768-d55516f4e3cb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.524591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d945f1-d1c8-486e-8768-d55516f4e3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.524859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d945f1-d1c8-486e-8768-d55516f4e3cb None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.525464 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d945f1-d1c8-486e-8768-d55516f4e3cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.528060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d945f1-d1c8-486e-8768-d55516f4e3cb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.537450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 337/672] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/services => generated 1258 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:06.546451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd2064-a100-408a-b622-4791fb8b2cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.547085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd2064-a100-408a-b622-4791fb8b2cb7 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.547357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd2064-a100-408a-b622-4791fb8b2cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.547623 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd2064-a100-408a-b622-4791fb8b2cb7 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.548374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd2064-a100-408a-b622-4791fb8b2cb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.549845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd2064-a100-408a-b622-4791fb8b2cb7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.575287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 336/673] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:06 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:07.300049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede2439-56c0-43ef-84fc-61ad36f7e295 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.300576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede2439-56c0-43ef-84fc-61ad36f7e295 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.300576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede2439-56c0-43ef-84fc-61ad36f7e295 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.302154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 338/674] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:07.307781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035709c-8f43-4fa1-ad39-d7c06e508cee None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.308293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035709c-8f43-4fa1-ad39-d7c06e508cee None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.308381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035709c-8f43-4fa1-ad39-d7c06e508cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.327388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a035709c-8f43-4fa1-ad39-d7c06e508cee None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:07.328592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a035709c-8f43-4fa1-ad39-d7c06e508cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:07.394638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.417626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 337/675] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:07.509747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ad036-fbe5-4382-bdf3-bcba34fd42d5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.510431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ad036-fbe5-4382-bdf3-bcba34fd42d5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.510900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ad036-fbe5-4382-bdf3-bcba34fd42d5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.512676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 339/676] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:07.518253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1727939-ef9f-4d8c-b8c7-c99b7cf653ff None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.518778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1727939-ef9f-4d8c-b8c7-c99b7cf653ff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.519297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1727939-ef9f-4d8c-b8c7-c99b7cf653ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.538694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d1727939-ef9f-4d8c-b8c7-c99b7cf653ff None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:07.540292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d1727939-ef9f-4d8c-b8c7-c99b7cf653ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:07.607284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 338/677] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:07.688447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ad036-fbe5-4382-bdf3-bcba34fd42d5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.781229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.792357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ad036-fbe5-4382-bdf3-bcba34fd42d5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.827227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5c36a-ce2e-479b-aee9-e74226606fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:07.827847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5c36a-ce2e-479b-aee9-e74226606fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.828118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5c36a-ce2e-479b-aee9-e74226606fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.828383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5c36a-ce2e-479b-aee9-e74226606fb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.839174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5c36a-ce2e-479b-aee9-e74226606fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ccd5c36a-ce2e-479b-aee9-e74226606fb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:07.843997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:07.844964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:07.845806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 340/678] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:07.851174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1727939-ef9f-4d8c-b8c7-c99b7cf653ff None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.893323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5c36a-ce2e-479b-aee9-e74226606fb0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.964457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1727939-ef9f-4d8c-b8c7-c99b7cf653ff None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.987074 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.003218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.004155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.004570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.005079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.005894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.006386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.008173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.011512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5c36a-ce2e-479b-aee9-e74226606fb0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.017173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 339/679] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.026238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.056478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.057803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.058269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.058934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.060069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.062162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:08.074663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:08.075820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:08.076679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:08.079781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 341/680] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:08.085832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.104799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.157549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da532c78-33ce-46a2-a9f2-55ad8b7ee990 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.190483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a9383-ecf0-4534-8005-4cf869b88f34 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.201209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.202344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.202795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.203243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.218178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.220119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.225249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.225857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.226156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.226428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.227397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.227702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.229175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:08.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:08.240099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:08.241204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:08.241204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:08.241204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:08.241409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 342/681] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.241409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.247800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 340/682] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:08.250312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.294673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.325620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.349713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1b412-f2a7-46c7-85df-9a1331aaed4e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.382729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b37090-199f-4b37-a5a7-1d69fd426837 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.385730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.386848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.387610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.388609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.389483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.390031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.391762 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.403997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 343/683] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.405317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.424650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.425333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.425626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.425899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.426582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.426886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.428546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.449925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 341/684] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.451895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.533833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301fea8-70ca-4e33-ad71-aa569c73eb0c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.575145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.576077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.576582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.577029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.577922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] RBAC: Adding query filter params (interface=public, service_id=f70326524bca4522bb08badd7aee9e20, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.578447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.580018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.581407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.592391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbedd7-faef-4cbc-a785-1d41999b0671 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.592730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 344/685] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/endpoints?service_id=f70326524bca4522bb08badd7aee9e20&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.595407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.606403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.632530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.633948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.633948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.635071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.643259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.644847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:08.659012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:08.660040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:08.660948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:08.666137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 342/686] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:08.667409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.724028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125da17b-97e3-45f6-ac4a-3dfb9e508332 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.768084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68720994-4668-41cd-ac69-ef2286074008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.768856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720994-4668-41cd-ac69-ef2286074008 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.769525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720994-4668-41cd-ac69-ef2286074008 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.769525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720994-4668-41cd-ac69-ef2286074008 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.770319 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68720994-4668-41cd-ac69-ef2286074008 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.771840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68720994-4668-41cd-ac69-ef2286074008 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.786401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 345/687] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:16:08.798262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c754f21-78f4-4463-b52f-08496bfc0490 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.817960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8013489-1782-4d88-9ffc-de11dceff2f2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.818390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8013489-1782-4d88-9ffc-de11dceff2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.818725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8013489-1782-4d88-9ffc-de11dceff2f2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.820368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 346/688] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:08.826408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba187-ae96-4e60-a392-634b021fd18a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.826689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba187-ae96-4e60-a392-634b021fd18a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.826963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba187-ae96-4e60-a392-634b021fd18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.845312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba84893-2466-4974-b2f3-3ccf449a9d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.846270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84893-2466-4974-b2f3-3ccf449a9d0d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.846720 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84893-2466-4974-b2f3-3ccf449a9d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.847133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84893-2466-4974-b2f3-3ccf449a9d0d None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.848023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba84893-2466-4974-b2f3-3ccf449a9d0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.848477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba84893-2466-4974-b2f3-3ccf449a9d0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.850581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba84893-2466-4974-b2f3-3ccf449a9d0d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.856482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e66ba187-ae96-4e60-a392-634b021fd18a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:08.857708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e66ba187-ae96-4e60-a392-634b021fd18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:08.876206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 343/689] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.882823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f3170-ae6f-458c-89f6-f105a8a14daf None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.883082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f3170-ae6f-458c-89f6-f105a8a14daf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.883354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f3170-ae6f-458c-89f6-f105a8a14daf None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.884886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 344/690] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:08.885582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f3170-ae6f-458c-89f6-f105a8a14daf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.907052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.952300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.961035 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 347/691] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:08.962752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283660fb-fa69-445d-bbcc-d1054df30df2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.963039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283660fb-fa69-445d-bbcc-d1054df30df2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.963591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283660fb-fa69-445d-bbcc-d1054df30df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.988899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-283660fb-fa69-445d-bbcc-d1054df30df2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:08.990155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-283660fb-fa69-445d-bbcc-d1054df30df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:09.034227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f3170-ae6f-458c-89f6-f105a8a14daf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.074547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389a127-3b5a-42c8-8e98-113cad005f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.076234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389a127-3b5a-42c8-8e98-113cad005f7f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.076234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389a127-3b5a-42c8-8e98-113cad005f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.076516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389a127-3b5a-42c8-8e98-113cad005f7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.089523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 348/692] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:09.158300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389a127-3b5a-42c8-8e98-113cad005f7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.159900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389a127-3b5a-42c8-8e98-113cad005f7f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.174682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.266965 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 345/693] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:09.272381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389a127-3b5a-42c8-8e98-113cad005f7f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.272714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283660fb-fa69-445d-bbcc-d1054df30df2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.382006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283660fb-fa69-445d-bbcc-d1054df30df2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.385219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389a127-3b5a-42c8-8e98-113cad005f7f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.400870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.414761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.421794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.422533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.422810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.423080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.423661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.425020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.429760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.433040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.433328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.433597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:09.434065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:09.434943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:09.435819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 349/694] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:09.436589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.447828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.449263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:09.465885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:09.467071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:09.467963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:09.467963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:09.467963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:09.467963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 346/695] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:09.467963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.551600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2722cfa-b7a8-43e6-a3e1-83dfcc322cdd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.584749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8606e3-d0e5-45b3-8aa4-907c7e1f1075 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.587048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.587689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.587963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.588235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.588826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.590230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:09.599185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:09.600186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:09.601006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 350/696] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:09.601878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.631219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.631944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.632287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.632623 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.633450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:09.633755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.635184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.644876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 347/697] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.644876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.672348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.672529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.715969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5154bf-a6c4-49ac-ac65-77d57a0e0cff None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.750621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89081d9-4622-4c61-8c3a-ae089f21bddd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.752540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.753140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.753416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.753685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.754356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:09.754664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.756020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.772527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 351/698] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.772527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.793915 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.794545 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.794822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.795094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.795727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:09.796027 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.797389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.802111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 348/699] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.802818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.879224 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ff70c-9f9d-4b8f-b8cc-f280d1a6af2a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.900841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.903584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.910819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d144d1-f3e0-433a-99ef-7dc2bc91f0ae None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.924411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.925237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.925736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.925982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.926685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:09.926949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.928343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.938037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 352/700] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.938037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.951147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.951782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.952055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.952320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.959509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.961036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:09.974413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:09.975335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:09.976222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:09.976222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:09.976222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:09.976222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:09.976222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:09.976222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:09.976222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:09.976222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:09.980371 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 349/701] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:09.981197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:10.043365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dd380-468c-41bf-aa44-cf0cb1beb72b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.076809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.077742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.078286 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.078655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.079362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:10.079582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.080940 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.086241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 353/702] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:10.087350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:10.089149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ebcb-d4ed-4f26-b1ff-1b29ca4a96ab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.126999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2347c7-7d88-468b-9600-bb36f78b3d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.127630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2347c7-7d88-468b-9600-bb36f78b3d59 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.127911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2347c7-7d88-468b-9600-bb36f78b3d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.128183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2347c7-7d88-468b-9600-bb36f78b3d59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.128851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2347c7-7d88-468b-9600-bb36f78b3d59 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.130258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2347c7-7d88-468b-9600-bb36f78b3d59 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.141133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:10.151755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 350/703] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:10.152833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2347c7-7d88-468b-9600-bb36f78b3d59 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:10.173320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:10.233290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a60a46-9ede-4592-9582-5bcb82f996cc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.292678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.293403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.293743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.294351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.295146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:10.295988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.297836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.299645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2347c7-7d88-468b-9600-bb36f78b3d59 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.318271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 354/704] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:10.321591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:10.347907 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.348451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.348729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.349056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.349766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] RBAC: Adding query filter params (interface=public, service_id=ab24855aab7c441a98b64eb34147a582, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:10.350074 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.351543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.358273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 351/705] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:10 2022] GET /identity/v3/endpoints?service_id=ab24855aab7c441a98b64eb34147a582&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:10.364244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:10.444703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2a150-9ac1-4f5c-a574-69328b8cf009 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.446567 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:10.451606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:10.488519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc8ad-19ac-45d3-9453-74202aa0d05f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.493620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07947b-5f51-4dc7-99ef-9601873a7256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.494267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07947b-5f51-4dc7-99ef-9601873a7256 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.494547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07947b-5f51-4dc7-99ef-9601873a7256 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.494819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07947b-5f51-4dc7-99ef-9601873a7256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.495551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07947b-5f51-4dc7-99ef-9601873a7256 None admin] RBAC: Adding query filter params (role.id=f049cc06faaa41d99bb5f9f49c1d1096, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=55fd5764905e4dfa899a90270181b220) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:10.495849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07947b-5f51-4dc7-99ef-9601873a7256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.497279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07947b-5f51-4dc7-99ef-9601873a7256 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.506200 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 355/706] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:10 2022] GET /identity/v3/role_assignments?user.id=55fd5764905e4dfa899a90270181b220&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=f049cc06faaa41d99bb5f9f49c1d1096 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:10.531402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f72923-11ed-46d1-8ab3-261c4373e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.532379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f72923-11ed-46d1-8ab3-261c4373e706 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.532848 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f72923-11ed-46d1-8ab3-261c4373e706 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.533297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f72923-11ed-46d1-8ab3-261c4373e706 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.534166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f72923-11ed-46d1-8ab3-261c4373e706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.535802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f72923-11ed-46d1-8ab3-261c4373e706 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.543756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 352/707] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:10 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:11.061852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cabf3-58e2-4e35-ba5d-fd97aa1a3dd2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.062268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cabf3-58e2-4e35-ba5d-fd97aa1a3dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.062447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cabf3-58e2-4e35-ba5d-fd97aa1a3dd2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.064043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 356/708] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:11.069250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9cff0-878e-4117-bdf1-930c821625bd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.069535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9cff0-878e-4117-bdf1-930c821625bd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.069810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9cff0-878e-4117-bdf1-930c821625bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.088789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-42e9cff0-878e-4117-bdf1-930c821625bd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:11.089969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-42e9cff0-878e-4117-bdf1-930c821625bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:11.166170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 353/709] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:11.437217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cabf3-58e2-4e35-ba5d-fd97aa1a3dd2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.549107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cabf3-58e2-4e35-ba5d-fd97aa1a3dd2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.584630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.585250 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.585526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.585822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.586776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:11.588146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:11.597046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:11.598006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:11.599039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 357/710] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:11.604649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9cff0-878e-4117-bdf1-930c821625bd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.670686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:11.705753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9cff0-878e-4117-bdf1-930c821625bd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.739869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.740728 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.741139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.741542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.742383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:11.742839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:11.744368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:11.749675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 354/711] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:11.755492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.788382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:11.863657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67a0b4-5d53-44ac-a0da-0d55908707b5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.899737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51a279-ea97-45ca-a462-a8cfbce73d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.900582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51a279-ea97-45ca-a462-a8cfbce73d50 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.901021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51a279-ea97-45ca-a462-a8cfbce73d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.901434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51a279-ea97-45ca-a462-a8cfbce73d50 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.902240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51a279-ea97-45ca-a462-a8cfbce73d50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:11.902700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51a279-ea97-45ca-a462-a8cfbce73d50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:11.904196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51a279-ea97-45ca-a462-a8cfbce73d50 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:11.910284 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 358/712] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:11.915494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.014408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ba39d-243e-4368-ba44-828dccd9dab0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.046592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.049238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048a32e-fc3e-4d7d-bcfd-9b3b39ed78ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.050108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a32e-fc3e-4d7d-bcfd-9b3b39ed78ca None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.050641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a32e-fc3e-4d7d-bcfd-9b3b39ed78ca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.050918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a32e-fc3e-4d7d-bcfd-9b3b39ed78ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.051812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048a32e-fc3e-4d7d-bcfd-9b3b39ed78ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.053347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048a32e-fc3e-4d7d-bcfd-9b3b39ed78ca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.075257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 355/713] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:12.471603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311b907-be4d-4b09-935f-970e9054958a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.471969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311b907-be4d-4b09-935f-970e9054958a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.472276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311b907-be4d-4b09-935f-970e9054958a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.473918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 359/714] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:12.480004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44794f59-4d63-41c9-b052-688f43f3d1c3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.480296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44794f59-4d63-41c9-b052-688f43f3d1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.480576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44794f59-4d63-41c9-b052-688f43f3d1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.499432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-44794f59-4d63-41c9-b052-688f43f3d1c3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:12.500674 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-44794f59-4d63-41c9-b052-688f43f3d1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:12.582220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 356/715] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:12.742209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23e5d-4cc5-4755-93d4-a2e473e13818 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.742844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23e5d-4cc5-4755-93d4-a2e473e13818 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.742844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23e5d-4cc5-4755-93d4-a2e473e13818 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.744277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 360/716] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:12.749755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a291c0c-980b-46b9-ad9f-c3494282511b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.750274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a291c0c-980b-46b9-ad9f-c3494282511b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.750439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a291c0c-980b-46b9-ad9f-c3494282511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.769068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2a291c0c-980b-46b9-ad9f-c3494282511b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:12.770276 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2a291c0c-980b-46b9-ad9f-c3494282511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:12.778205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e1e6-f5b5-4bda-985c-04594dc74c4e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.778794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e1e6-f5b5-4bda-985c-04594dc74c4e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.779269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e1e6-f5b5-4bda-985c-04594dc74c4e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.781048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 361/717] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:12.786245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014aa4-f82d-43ba-b04d-6f15d408b557 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.786760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014aa4-f82d-43ba-b04d-6f15d408b557 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.787225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014aa4-f82d-43ba-b04d-6f15d408b557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.806921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4e014aa4-f82d-43ba-b04d-6f15d408b557 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:12.808448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4e014aa4-f82d-43ba-b04d-6f15d408b557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:12.838928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 357/718] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:12.855092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a291c0c-980b-46b9-ad9f-c3494282511b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.882716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 362/719] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:12.912988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.973570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a291c0c-980b-46b9-ad9f-c3494282511b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.008837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.009476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.009757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.010031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.010669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.012075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:13.021780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:13.022828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:13.023706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 358/720] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:13.031219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014aa4-f82d-43ba-b04d-6f15d408b557 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.057019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.123883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.135667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014aa4-f82d-43ba-b04d-6f15d408b557 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.171880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.172478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.172751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.173015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.173735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] RBAC: Adding query filter params (name=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.174029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.175429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.180333 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 363/721] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:13.182122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.239579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8769-6616-4126-8bbe-2c76f44b47a6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.248114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.274774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.278081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.279265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.280005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.280451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] PATH_INFO: `/v3/services/image` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.281385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.283574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.287388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13310daf-2ac1-4331-9ad6-23e3fee9d9da None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:13.294898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:13.295889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:13.298681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 359/722] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:13.299637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.326832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.327463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.327743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.328013 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.328592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.329957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:13.339338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:13.340316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:13.343073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 364/723] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:13.343856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.407497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e84133-1998-4b1a-a1a3-707fa58c82bb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.441889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.442530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.442810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.443085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.443708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.444013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.445630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.448024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaa01c-8672-4f70-8060-c40fa0db5738 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.454298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 360/724] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:13.455134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.492533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.493537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.494012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.494600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.495401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.495900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.497682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.500691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.504810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 365/725] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:13.510414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.533610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.592184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188702c0-0b22-4783-9f5f-7f5c2719346c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.638122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.638122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.638122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.638122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.638556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.638556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.639070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.641476 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d88aad-90d6-4a0c-9fc0-e313212ec80c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.653178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 361/726] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:13.653744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.685509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.686288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.686644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.686913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.687673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.687955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.689337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.695564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 366/727] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:13.769446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.773881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d30b79-96ca-4ece-b491-bed1495ecd28 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.812703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f0467-6811-43e1-a835-e823652d1f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.813410 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0467-6811-43e1-a835-e823652d1f1b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.813780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0467-6811-43e1-a835-e823652d1f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.814233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0467-6811-43e1-a835-e823652d1f1b None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.814993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0467-6811-43e1-a835-e823652d1f1b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.815398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0467-6811-43e1-a835-e823652d1f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.816906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0467-6811-43e1-a835-e823652d1f1b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.823174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 362/728] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:13.829228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.929406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.949196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b6013-72af-4aab-922f-eeca7b7454b1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.984526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5545a-d26f-4547-a842-462253fdd80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.985289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5545a-d26f-4547-a842-462253fdd80c None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.985663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5545a-d26f-4547-a842-462253fdd80c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.986052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5545a-d26f-4547-a842-462253fdd80c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.986959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5545a-d26f-4547-a842-462253fdd80c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.988490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5545a-d26f-4547-a842-462253fdd80c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:14.014797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 367/729] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:14.289211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f36f-3d15-4cb1-9f29-04ea33d15abd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:14.289497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f36f-3d15-4cb1-9f29-04ea33d15abd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:14.289779 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f36f-3d15-4cb1-9f29-04ea33d15abd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:14.291403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 363/730] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:14.296964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ada51e-9d84-4c46-a4d9-3460deb42707 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:14.297249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ada51e-9d84-4c46-a4d9-3460deb42707 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:14.297516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ada51e-9d84-4c46-a4d9-3460deb42707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:14.315807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-27ada51e-9d84-4c46-a4d9-3460deb42707 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:14.316989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-27ada51e-9d84-4c46-a4d9-3460deb42707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:14.389977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 368/731] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:14.657497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7f36f-3d15-4cb1-9f29-04ea33d15abd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:14.763784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7f36f-3d15-4cb1-9f29-04ea33d15abd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:14.797621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:14.798292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:14.798532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:14.798792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:14.799373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:14.800726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:14.809611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:14.810646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:14.811461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 364/732] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:14.817153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ada51e-9d84-4c46-a4d9-3460deb42707 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:14.878773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:14.922335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ada51e-9d84-4c46-a4d9-3460deb42707 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:14.957190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:14.957799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:14.958069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:14.958365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:14.958992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:14.959289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:14.960647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:14.966236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 369/733] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:14.975708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:15.005830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:15.078813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b252475-0cf4-497a-9344-7c1887c8d60e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:15.113236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:15.114188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.114659 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.115145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.115971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:15.116468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:15.118151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:15.123491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 365/734] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:15.132732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:15.232501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b96ca-d55d-430a-99f9-f00af5dd8b0d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:15.257856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:15.267801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3abc6-9e35-4627-87ff-1f9d20331303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:15.268403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3abc6-9e35-4627-87ff-1f9d20331303 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.268677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3abc6-9e35-4627-87ff-1f9d20331303 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.268946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3abc6-9e35-4627-87ff-1f9d20331303 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.269646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3abc6-9e35-4627-87ff-1f9d20331303 None admin] RBAC: Adding query filter params (interface=admin, service_id=f70326524bca4522bb08badd7aee9e20, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:15.269954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3abc6-9e35-4627-87ff-1f9d20331303 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:15.271357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3abc6-9e35-4627-87ff-1f9d20331303 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:15.276982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 370/735] 10.222.0.25 () {56 vars in 1249 bytes} [Wed Nov 2 07:16:15 2022] GET /identity/v3/endpoints?service_id=f70326524bca4522bb08badd7aee9e20&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:15.282728 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:15.374085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:15.383616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2421b62-2e92-4f7f-bdee-da0db6ff011c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:15.418383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edf3e8-d38a-4f61-bc04-77750d77654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:15.418996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf3e8-d38a-4f61-bc04-77750d77654d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.419270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf3e8-d38a-4f61-bc04-77750d77654d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.419537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf3e8-d38a-4f61-bc04-77750d77654d None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.420139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edf3e8-d38a-4f61-bc04-77750d77654d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:15.421503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edf3e8-d38a-4f61-bc04-77750d77654d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:15.427921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 366/736] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:15 2022] GET /identity/v3/services => generated 1497 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:15.793784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ce59f-a786-4005-8207-a288d220924a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.794215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ce59f-a786-4005-8207-a288d220924a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.794411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ce59f-a786-4005-8207-a288d220924a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.795968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 371/737] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:15.801214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a5a52-48bf-4ace-8395-cb3c95afb0d0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.801682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a5a52-48bf-4ace-8395-cb3c95afb0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.801760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a5a52-48bf-4ace-8395-cb3c95afb0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.823113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b08a5a52-48bf-4ace-8395-cb3c95afb0d0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:15.824284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b08a5a52-48bf-4ace-8395-cb3c95afb0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:15.892509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 367/738] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:16.024277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6fcc2-b16d-4f93-9c56-2e0c8b3b5e74 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.025252 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6fcc2-b16d-4f93-9c56-2e0c8b3b5e74 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.025717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6fcc2-b16d-4f93-9c56-2e0c8b3b5e74 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.027542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 372/739] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:16.033057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c952aa-cec0-4988-a99a-6fa463cfbc30 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.033343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c952aa-cec0-4988-a99a-6fa463cfbc30 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.033616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c952aa-cec0-4988-a99a-6fa463cfbc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.052104 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-60c952aa-cec0-4988-a99a-6fa463cfbc30 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:16.053214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-60c952aa-cec0-4988-a99a-6fa463cfbc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:16.124751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 368/740] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:16.185859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6fcc2-b16d-4f93-9c56-2e0c8b3b5e74 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.300100 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6fcc2-b16d-4f93-9c56-2e0c8b3b5e74 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.323694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc13bc3-436c-40a5-886c-35b23f7f7b0a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.323994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc13bc3-436c-40a5-886c-35b23f7f7b0a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.324174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc13bc3-436c-40a5-886c-35b23f7f7b0a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.325723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 369/741] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:16.331686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b7df5-7126-4667-aa22-8c1e75a96a6a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.331974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b7df5-7126-4667-aa22-8c1e75a96a6a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.332248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b7df5-7126-4667-aa22-8c1e75a96a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.348732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f40c4-d722-4c76-993c-0fa95113ebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.350157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f40c4-d722-4c76-993c-0fa95113ebe4 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.350707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f40c4-d722-4c76-993c-0fa95113ebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.351147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f40c4-d722-4c76-993c-0fa95113ebe4 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.352245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f40c4-d722-4c76-993c-0fa95113ebe4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.353691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7c9b7df5-7126-4667-aa22-8c1e75a96a6a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:16.354199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f40c4-d722-4c76-993c-0fa95113ebe4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.354978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7c9b7df5-7126-4667-aa22-8c1e75a96a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:16.367295 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.377624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 373/742] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:16 2022] POST /identity/v3/services => generated 242 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:16.433139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f40c4-d722-4c76-993c-0fa95113ebe4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.450145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 370/743] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:16.498309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.557106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f40c4-d722-4c76-993c-0fa95113ebe4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.591848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154f6d-ea15-4ddc-b0af-d4b41b716c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.592510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154f6d-ea15-4ddc-b0af-d4b41b716c1f None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.592794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154f6d-ea15-4ddc-b0af-d4b41b716c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.593062 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154f6d-ea15-4ddc-b0af-d4b41b716c1f None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.593720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154f6d-ea15-4ddc-b0af-d4b41b716c1f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.595120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154f6d-ea15-4ddc-b0af-d4b41b716c1f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.611975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 374/744] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:16 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:16.720402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b7df5-7126-4667-aa22-8c1e75a96a6a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.826992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b7df5-7126-4667-aa22-8c1e75a96a6a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.870432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.871096 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.871373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.871637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.872264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.873690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.876312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING keystone.server.flask.common [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 07:16:16.880817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.895551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.sql.core [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:16.895551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:16.895551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [parameters: {'id': '1ad254ebf7a4469680237d67baf0706d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 07:16:16.895551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1ad254ebf7a4469680237d67baf0706d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1ad254ebf7a4469680237d67baf0706d. Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:16.903917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:16:16.904705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application yield resource Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise self.value Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application session.commit() Nov 02 07:16:16.905637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:16:16.907030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:16:16.907916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:16.908706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [parameters: {'id': '1ad254ebf7a4469680237d67baf0706d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.909554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:16.910367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1ad254ebf7a4469680237d67baf0706d. Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:16.911235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 371/745] 10.222.0.25 () {60 vars in 1140 bytes} [Wed Nov 2 07:16:16 2022] POST /identity/v3/projects => generated 262 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:16.915235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154f6d-ea15-4ddc-b0af-d4b41b716c1f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.027850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154f6d-ea15-4ddc-b0af-d4b41b716c1f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.039549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.064596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765d686-6e2b-4cdc-8fe7-1d499b3ccc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.065263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d686-6e2b-4cdc-8fe7-1d499b3ccc3d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.065539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d686-6e2b-4cdc-8fe7-1d499b3ccc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.065805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d686-6e2b-4cdc-8fe7-1d499b3ccc3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.072872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765d686-6e2b-4cdc-8fe7-1d499b3ccc3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.074338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765d686-6e2b-4cdc-8fe7-1d499b3ccc3d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d765d686-6e2b-4cdc-8fe7-1d499b3ccc3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:17.085564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:17.087037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:17.087898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 375/746] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:17.094860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.199704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce4d1b-444d-4bf6-9e00-b3a4164a4662 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.237326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aaa2e3-9b81-48e9-aa20-ffa9aafc818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.238381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaa2e3-9b81-48e9-aa20-ffa9aafc818d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.238868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaa2e3-9b81-48e9-aa20-ffa9aafc818d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.239324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaa2e3-9b81-48e9-aa20-ffa9aafc818d None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.240220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aaa2e3-9b81-48e9-aa20-ffa9aafc818d None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:17.240708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aaa2e3-9b81-48e9-aa20-ffa9aafc818d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.242618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aaa2e3-9b81-48e9-aa20-ffa9aafc818d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.263719 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 372/747] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:17 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:17.674202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d9c67-eb40-4636-8c32-48a517b2e8d1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.674911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d9c67-eb40-4636-8c32-48a517b2e8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.675385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d9c67-eb40-4636-8c32-48a517b2e8d1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.677183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 376/748] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:17.683785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c0208-7661-4aea-ad32-fdc8155139ba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.684380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c0208-7661-4aea-ad32-fdc8155139ba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.684857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c0208-7661-4aea-ad32-fdc8155139ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.704234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ec5c0208-7661-4aea-ad32-fdc8155139ba None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:17.705740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ec5c0208-7661-4aea-ad32-fdc8155139ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:17.755549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.776530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 373/749] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:18.050502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d9c67-eb40-4636-8c32-48a517b2e8d1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.157927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d9c67-eb40-4636-8c32-48a517b2e8d1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.191451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.192081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.192358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.192633 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.193204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.194705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:18.204096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:18.205067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:18.205872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 377/750] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:18.215128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c0208-7661-4aea-ad32-fdc8155139ba None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.317696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c0208-7661-4aea-ad32-fdc8155139ba None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.353290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.354287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.354756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.355238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.356120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:18.356607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.358189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.363751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 374/751] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:18.369930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.404847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:18.471317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1514a08-1e78-4eff-a5fa-45a7ecbfd718 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.509437 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92938e77-208c-40bd-ba4a-8a22b8482f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.510123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938e77-208c-40bd-ba4a-8a22b8482f70 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.510482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938e77-208c-40bd-ba4a-8a22b8482f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.510753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938e77-208c-40bd-ba4a-8a22b8482f70 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.511389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92938e77-208c-40bd-ba4a-8a22b8482f70 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:18.511678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92938e77-208c-40bd-ba4a-8a22b8482f70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.513046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92938e77-208c-40bd-ba4a-8a22b8482f70 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.518682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 378/752] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:18.524263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.575196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:18.636299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278d0a9-beb2-48a8-bebe-b169777cce2c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.671558 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d59c04-f9eb-43b7-8d26-43c47af0ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.672167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59c04-f9eb-43b7-8d26-43c47af0ea56 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.672439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59c04-f9eb-43b7-8d26-43c47af0ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.672708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59c04-f9eb-43b7-8d26-43c47af0ea56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.673378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d59c04-f9eb-43b7-8d26-43c47af0ea56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.674770 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d59c04-f9eb-43b7-8d26-43c47af0ea56 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.696357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 375/753] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:18.730378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f494-7eaa-4d74-a666-daa2a59cdfab None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.730718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f494-7eaa-4d74-a666-daa2a59cdfab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.730936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f494-7eaa-4d74-a666-daa2a59cdfab None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.732650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 379/754] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:18.738542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f1e9-6552-4b49-8e02-75175622a455 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.738869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f1e9-6552-4b49-8e02-75175622a455 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.739146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f1e9-6552-4b49-8e02-75175622a455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.759466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1fe4f1e9-6552-4b49-8e02-75175622a455 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:18.760702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1fe4f1e9-6552-4b49-8e02-75175622a455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:18.848504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 376/755] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:18 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:18.900282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78b560-a4f9-487f-bf45-e519da052ba2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.900573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78b560-a4f9-487f-bf45-e519da052ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.900849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78b560-a4f9-487f-bf45-e519da052ba2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.902363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 380/756] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:18.908041 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac00a-2b01-418e-a8bf-e2f4a93f3701 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.908326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac00a-2b01-418e-a8bf-e2f4a93f3701 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.908606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac00a-2b01-418e-a8bf-e2f4a93f3701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.927347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-df6ac00a-2b01-418e-a8bf-e2f4a93f3701 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:18.928511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-df6ac00a-2b01-418e-a8bf-e2f4a93f3701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:18.956985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.000080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 377/757] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:18 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:19.120933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78b560-a4f9-487f-bf45-e519da052ba2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.223581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.230751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78b560-a4f9-487f-bf45-e519da052ba2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.266138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.266769 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.267043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.267315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.267888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.268539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac00a-2b01-418e-a8bf-e2f4a93f3701 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.269289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:19.280539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:19.281552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:19.282926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 381/758] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:19.289734 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.377493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac00a-2b01-418e-a8bf-e2f4a93f3701 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.417983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df199da5-3707-4e7a-b5af-c23479329215 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.434033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.434701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.434980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.435248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] PATH_INFO: `/v3/services/image` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.436107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.437805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:19.450616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:19.451669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:19.457199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 378/759] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:19.459482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.462022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.463287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.463752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.465317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.466351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:19.466893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.468482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.477005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 382/760] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:19.483143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.496705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.517001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.585857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119ae2d-d6f7-41e4-899a-e54ef96d8f42 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.614610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b1dfe-7605-40ce-9ad1-9f859b5c766c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.620116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82904794-7610-4d27-b5de-cca24670cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.620724 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904794-7610-4d27-b5de-cca24670cffe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.621000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904794-7610-4d27-b5de-cca24670cffe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.621270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904794-7610-4d27-b5de-cca24670cffe None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.621938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82904794-7610-4d27-b5de-cca24670cffe None admin] RBAC: Adding query filter params (name=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:19.622272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82904794-7610-4d27-b5de-cca24670cffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.623923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82904794-7610-4d27-b5de-cca24670cffe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.628568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 379/761] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:19.629815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c195db-d569-485e-bb95-cc2b039535d6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.630116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c195db-d569-485e-bb95-cc2b039535d6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.630405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c195db-d569-485e-bb95-cc2b039535d6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.632060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 380/762] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:19.634815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c195db-d569-485e-bb95-cc2b039535d6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.657545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c18fa-2597-46d5-bde8-23a4bb521397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.658459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c18fa-2597-46d5-bde8-23a4bb521397 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.659429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c18fa-2597-46d5-bde8-23a4bb521397 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.659863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c18fa-2597-46d5-bde8-23a4bb521397 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.660928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c18fa-2597-46d5-bde8-23a4bb521397 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:19.661399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c18fa-2597-46d5-bde8-23a4bb521397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.663844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c18fa-2597-46d5-bde8-23a4bb521397 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.670155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 383/763] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:19.671345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856316f-c396-4b07-9736-f8dbcf88d106 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.671696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856316f-c396-4b07-9736-f8dbcf88d106 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.672056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856316f-c396-4b07-9736-f8dbcf88d106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.690209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9856316f-c396-4b07-9736-f8dbcf88d106 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:19.691692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9856316f-c396-4b07-9736-f8dbcf88d106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:19.737996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.750080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c195db-d569-485e-bb95-cc2b039535d6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.759955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 384/764] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:19.761030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856316f-c396-4b07-9736-f8dbcf88d106 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.790739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.791503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.791892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.792253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.792968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] RBAC: Adding query filter params (type=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:19.793360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.794697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.795700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.802255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 381/765] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:19.809031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.879845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856316f-c396-4b07-9736-f8dbcf88d106 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.919703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.920524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.920997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.921437 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.922333 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] RBAC: Adding query filter params (interface=public, service_id=528a1aa5aa5d4d3f98a6ac64f0669801, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:19.922814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.924342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.927512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79772-1440-48cb-82b6-97338bf14b9b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.934502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 385/766] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/endpoints?service_id=528a1aa5aa5d4d3f98a6ac64f0669801&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:19.936396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.968248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.968869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.969172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.969588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.970318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] RBAC: Adding query filter params (interface=public, service_id=532c4f5579c7476d809145061834eafb, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:19.970900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.972097 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.982190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 382/767] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/endpoints?service_id=532c4f5579c7476d809145061834eafb&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:19.986947 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.012069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.039076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.057673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e268f08-9887-4f48-8741-c8dcd0d43fab None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.091753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be803508-6f50-4327-97d7-f922e0b85575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.092358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803508-6f50-4327-97d7-f922e0b85575 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.092638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803508-6f50-4327-97d7-f922e0b85575 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.092913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803508-6f50-4327-97d7-f922e0b85575 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.093511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be803508-6f50-4327-97d7-f922e0b85575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.094927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be803508-6f50-4327-97d7-f922e0b85575 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.098967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d38324-cc4a-44cb-84cd-180b91a510ab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.101577 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 386/768] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:20.102378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be803508-6f50-4327-97d7-f922e0b85575 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.147848 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dff66-336e-4840-b5de-d49cc5bffafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.148424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dff66-336e-4840-b5de-d49cc5bffafd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.148702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dff66-336e-4840-b5de-d49cc5bffafd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.148971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dff66-336e-4840-b5de-d49cc5bffafd None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.149584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dff66-336e-4840-b5de-d49cc5bffafd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.151053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dff66-336e-4840-b5de-d49cc5bffafd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.162627 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 383/769] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services => generated 1957 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:20.228044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be803508-6f50-4327-97d7-f922e0b85575 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.267623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.268255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.268527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.268793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.269444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.270933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.273748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING keystone.server.flask.common [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 07:16:20.288660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:16:20.288660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 07:16:20.288660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [parameters: {'user_id': 'dc5c9b9c9b984b7f9bb1c75dce48e59e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 07:16:20.288660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:20.296765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:20.297723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:16:20.299185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:16:20.301278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = connection._execute_20( Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:20.304475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:20.309701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dc5c9b9c9b984b7f9bb1c75dce48e59e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:20.312923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Nov 02 07:16:20.314854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Nov 02 07:16:20.316980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Nov 02 07:16:20.316980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Nov 02 07:16:20.316980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:16:20.316980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:16:20.316980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 07:16:20.316980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:20.316980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 387/770] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Nov 2 07:16:20 2022] POST /identity/v3/users => generated 160 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:20.318767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dff66-336e-4840-b5de-d49cc5bffafd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.403305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.422962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dff66-336e-4840-b5de-d49cc5bffafd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.457976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2807dd6-9ce9-4635-be9e-b27a49a67824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.458989 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2807dd6-9ce9-4635-be9e-b27a49a67824 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.459466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2807dd6-9ce9-4635-be9e-b27a49a67824 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.459976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2807dd6-9ce9-4635-be9e-b27a49a67824 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.469698 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2807dd6-9ce9-4635-be9e-b27a49a67824 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.471488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2807dd6-9ce9-4635-be9e-b27a49a67824 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b2807dd6-9ce9-4635-be9e-b27a49a67824 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:20.483489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:20.484438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:16:20.485365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:20.488017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 384/771] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:20.494146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.510695 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.602671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345c081-6e38-4ff4-a01c-32495115634a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.636997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e94cc-ad39-41f8-8084-24e7478efff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.637598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e94cc-ad39-41f8-8084-24e7478efff1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.637875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e94cc-ad39-41f8-8084-24e7478efff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.638168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e94cc-ad39-41f8-8084-24e7478efff1 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.638862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e94cc-ad39-41f8-8084-24e7478efff1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:20.639157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e94cc-ad39-41f8-8084-24e7478efff1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.640775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e94cc-ad39-41f8-8084-24e7478efff1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.681069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 388/772] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.941855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d2906-b84a-482a-adb8-877965f9f6f3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.942032 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d2906-b84a-482a-adb8-877965f9f6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.942340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d2906-b84a-482a-adb8-877965f9f6f3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.943969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 385/773] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:20.949909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9ef52-5086-4f56-80ca-f42ad04dc15a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.950225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9ef52-5086-4f56-80ca-f42ad04dc15a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.950497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9ef52-5086-4f56-80ca-f42ad04dc15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.970234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-33f9ef52-5086-4f56-80ca-f42ad04dc15a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:20.971763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33f9ef52-5086-4f56-80ca-f42ad04dc15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:21.032159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.038060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 389/774] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:21.299834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d2906-b84a-482a-adb8-877965f9f6f3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.406626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d2906-b84a-482a-adb8-877965f9f6f3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.440520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541e7bd-889f-4e19-9ed6-48c5e59d3a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.441156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e7bd-889f-4e19-9ed6-48c5e59d3a6a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.441431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e7bd-889f-4e19-9ed6-48c5e59d3a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.441698 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e7bd-889f-4e19-9ed6-48c5e59d3a6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.442838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541e7bd-889f-4e19-9ed6-48c5e59d3a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.444273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541e7bd-889f-4e19-9ed6-48c5e59d3a6a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.446058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 386/775] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:21.451762 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9ef52-5086-4f56-80ca-f42ad04dc15a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.553348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9ef52-5086-4f56-80ca-f42ad04dc15a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.587009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc1a43-4f97-497e-87a5-ead4152e5529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.587618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc1a43-4f97-497e-87a5-ead4152e5529 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.587891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc1a43-4f97-497e-87a5-ead4152e5529 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.588152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc1a43-4f97-497e-87a5-ead4152e5529 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.588786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc1a43-4f97-497e-87a5-ead4152e5529 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.590231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc1a43-4f97-497e-87a5-ead4152e5529 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.626120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 390/776] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:21 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:22.375256 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1921-c732-4aa1-be7a-4b43e4964aef None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.375640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1921-c732-4aa1-be7a-4b43e4964aef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.375817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1921-c732-4aa1-be7a-4b43e4964aef None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.377195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 387/777] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:22.382786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae59e5-b1b2-4624-811f-9991e8b7a888 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.383052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae59e5-b1b2-4624-811f-9991e8b7a888 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.383499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae59e5-b1b2-4624-811f-9991e8b7a888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.402514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-65ae59e5-b1b2-4624-811f-9991e8b7a888 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:22.403684 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-65ae59e5-b1b2-4624-811f-9991e8b7a888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:22.404833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50627d6-3426-4e43-9224-3ddaaf32f2ba None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.405117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50627d6-3426-4e43-9224-3ddaaf32f2ba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.405394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50627d6-3426-4e43-9224-3ddaaf32f2ba None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.406730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 388/778] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:22.411853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca221fc8-c33d-4863-abcb-c3f4d60ebcc0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.412098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca221fc8-c33d-4863-abcb-c3f4d60ebcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.412400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca221fc8-c33d-4863-abcb-c3f4d60ebcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.431067 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ca221fc8-c33d-4863-abcb-c3f4d60ebcc0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:22.432234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ca221fc8-c33d-4863-abcb-c3f4d60ebcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:22.436702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:22.453692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:22.477514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 391/779] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:22 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:22.509000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 389/780] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:22 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:22.742018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae59e5-b1b2-4624-811f-9991e8b7a888 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:22.770983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca221fc8-c33d-4863-abcb-c3f4d60ebcc0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:22.884984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca221fc8-c33d-4863-abcb-c3f4d60ebcc0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:22.928264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:22.928912 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.929507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.929858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.930485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:22.931860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:22.941215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:22.942225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:22.947550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 390/781] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Nov 2 07:16:22 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:22.949758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:22.953525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:22.971815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae59e5-b1b2-4624-811f-9991e8b7a888 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.007862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.008605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.008883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.009158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.009746 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.011185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:23.020593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:23.021601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:23.022885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 392/782] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:23.027717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.059988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf749a-e870-4e13-a3b9-b3b78e3889c8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.076421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:23.094816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.095422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.095701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.095975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.096606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] RBAC: Adding query filter params (name=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:23.096905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.098296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.104167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 391/783] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:23.109852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.131793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301d68b-dfbb-40d4-bd89-3e1b950c7aaf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.166281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.166878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.167155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.167424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.168048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:23.168346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.169709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.179545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 393/784] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:23.180838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.183148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:23.219315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65d6d5-76e4-4416-b7be-9ff02b5d9992 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.256461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.257139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.257386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.257918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.258532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] RBAC: Adding query filter params (type=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:23.258839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.260213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.265565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 392/785] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:23.271258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.284335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f957bf-2bf1-4034-b624-16e127281778 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.294269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:23.321412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.322298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.322578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.322854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.323473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:23.323772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.325129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.330583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 394/786] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:23.336317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.378951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200d4ac-a0da-4b56-85bd-740812c8650c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.412311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:23.420093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf258e2-da32-4444-8100-f77042ca6373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.420725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf258e2-da32-4444-8100-f77042ca6373 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.421004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf258e2-da32-4444-8100-f77042ca6373 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.421277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf258e2-da32-4444-8100-f77042ca6373 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.422232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf258e2-da32-4444-8100-f77042ca6373 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.423998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf258e2-da32-4444-8100-f77042ca6373 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.438131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138543d-5ec1-4a98-afb7-3621e8e8ddff None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.450004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 393/787] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:23.487976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe0106-86cb-41fc-abb0-34aa77434678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.488614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0106-86cb-41fc-abb0-34aa77434678 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.488859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0106-86cb-41fc-abb0-34aa77434678 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.489119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0106-86cb-41fc-abb0-34aa77434678 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.489805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe0106-86cb-41fc-abb0-34aa77434678 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.491221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe0106-86cb-41fc-abb0-34aa77434678 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.518024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 395/788] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:23.817784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e2c80-8de1-44d9-b8ab-11a3fbce53a0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.817978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e2c80-8de1-44d9-b8ab-11a3fbce53a0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.818290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e2c80-8de1-44d9-b8ab-11a3fbce53a0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.819785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 394/789] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:23.825600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3014861-c079-44de-8b07-ccaa4817c517 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.825880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3014861-c079-44de-8b07-ccaa4817c517 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.826180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3014861-c079-44de-8b07-ccaa4817c517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.845083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f3014861-c079-44de-8b07-ccaa4817c517 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:23.846280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f3014861-c079-44de-8b07-ccaa4817c517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:23.863988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:23.933545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 396/790] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:24.204175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e2c80-8de1-44d9-b8ab-11a3fbce53a0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:24.313183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e2c80-8de1-44d9-b8ab-11a3fbce53a0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:24.348693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29424213-77ee-46d9-b67a-72592b47d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:24.349331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29424213-77ee-46d9-b67a-72592b47d35a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.349606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29424213-77ee-46d9-b67a-72592b47d35a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.349869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29424213-77ee-46d9-b67a-72592b47d35a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.351003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29424213-77ee-46d9-b67a-72592b47d35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:24.352599 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29424213-77ee-46d9-b67a-72592b47d35a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:24.354356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 395/791] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:24.360194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3014861-c079-44de-8b07-ccaa4817c517 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:24.461871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3014861-c079-44de-8b07-ccaa4817c517 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:24.497846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c83b6-e77d-4da9-aa45-1b36f2926dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:24.498542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c83b6-e77d-4da9-aa45-1b36f2926dd2 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.498766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c83b6-e77d-4da9-aa45-1b36f2926dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.499015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c83b6-e77d-4da9-aa45-1b36f2926dd2 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.499671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c83b6-e77d-4da9-aa45-1b36f2926dd2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:24.501111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c83b6-e77d-4da9-aa45-1b36f2926dd2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:24.524041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 397/792] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:24 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:25.627491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f75e1-bb73-4b48-9d98-dab7a19b468f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.628103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f75e1-bb73-4b48-9d98-dab7a19b468f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.628103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f75e1-bb73-4b48-9d98-dab7a19b468f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.629360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 396/793] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:25.635601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfee83-65e1-4eab-8c32-a57505b51e9e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.636140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfee83-65e1-4eab-8c32-a57505b51e9e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.636140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfee83-65e1-4eab-8c32-a57505b51e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.656713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.662837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c7dfee83-65e1-4eab-8c32-a57505b51e9e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:25.664790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c7dfee83-65e1-4eab-8c32-a57505b51e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:25.736383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 398/794] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:25.757843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e912f1-acff-46a7-90e4-36be5706fa06 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.758078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e912f1-acff-46a7-90e4-36be5706fa06 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.758422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e912f1-acff-46a7-90e4-36be5706fa06 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.759934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 397/795] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:25.765030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec66a1-bc75-4740-8d0b-c65977adf2fd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.765328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec66a1-bc75-4740-8d0b-c65977adf2fd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.765550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec66a1-bc75-4740-8d0b-c65977adf2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.784463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8eec66a1-bc75-4740-8d0b-c65977adf2fd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:25.785640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8eec66a1-bc75-4740-8d0b-c65977adf2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:25.854771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 399/796] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:26.002760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e912f1-acff-46a7-90e4-36be5706fa06 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.061790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.108030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e912f1-acff-46a7-90e4-36be5706fa06 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.121025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec66a1-bc75-4740-8d0b-c65977adf2fd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.145550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.146202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.146464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.146742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.147452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.149096 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.157268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.sql.core [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:26.157268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:26.157268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [parameters: {'id': '8ad7fb477a474c0b9c493034d8ecfb56', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:16:26.157268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:26.164556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:16:26.165436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application yield resource Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise self.value Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application session.commit() Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:16:26.166777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:16:26.167667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:16:26.168647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:26.169417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [parameters: {'id': '8ad7fb477a474c0b9c493034d8ecfb56', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:16:26.171236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:26.173285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 398/797] 10.222.0.25 () {60 vars in 1137 bytes} [Wed Nov 2 07:16:26 2022] POST /identity/v3/domains => generated 190 bytes in 164 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:26.175603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.201133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.237673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec66a1-bc75-4740-8d0b-c65977adf2fd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.275306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ade52-8f30-4032-a57c-dc4b8a86332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.276173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ade52-8f30-4032-a57c-dc4b8a86332e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.276592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ade52-8f30-4032-a57c-dc4b8a86332e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.276998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ade52-8f30-4032-a57c-dc4b8a86332e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.280235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc674237-064d-4eb0-a011-5d4e1b096aaa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.287585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ade52-8f30-4032-a57c-dc4b8a86332e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-db3ade52-8f30-4032-a57c-dc4b8a86332e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:26.292730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:26.293654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:26.294503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 400/798] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:26.300280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ade52-8f30-4032-a57c-dc4b8a86332e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.303468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.319694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.320418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.320900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.321177 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.332625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.334360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:26.349689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:26.350903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:26.351987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:26.351987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:26.351987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:26.357494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 399/799] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:26.358458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.407719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ade52-8f30-4032-a57c-dc4b8a86332e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.433974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.444283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.444888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.445170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.445436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.446121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.446456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.447919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.456766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 401/800] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:26.463290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.466078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db6f57-b4c3-4602-8ffb-3eb4d6ad3f2a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.501824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.502470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.502749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.503021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.504136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.504449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.506235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.518983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 400/801] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.584797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c8ea7-6b75-469e-8f77-dc45416b919e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.625856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c75ac-0030-4e36-a8aa-9b1e63f4ba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.626537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c75ac-0030-4e36-a8aa-9b1e63f4ba86 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.626810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c75ac-0030-4e36-a8aa-9b1e63f4ba86 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.627073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c75ac-0030-4e36-a8aa-9b1e63f4ba86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.639129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c75ac-0030-4e36-a8aa-9b1e63f4ba86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.640596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c75ac-0030-4e36-a8aa-9b1e63f4ba86 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0d0c75ac-0030-4e36-a8aa-9b1e63f4ba86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:26.656468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:26.657525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:26.658477 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:26.658477 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:26.658477 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:26.664227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 402/802] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:26.664416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.699302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.699665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c3f1-cd1a-4fbd-8b6d-e0e275991af8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.699996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c3f1-cd1a-4fbd-8b6d-e0e275991af8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.700312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c3f1-cd1a-4fbd-8b6d-e0e275991af8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.701978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 403/803] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:26.707273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeddf45-45bb-4458-837c-7e509f0a19a8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.707623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeddf45-45bb-4458-837c-7e509f0a19a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.707898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeddf45-45bb-4458-837c-7e509f0a19a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.726955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cfeddf45-45bb-4458-837c-7e509f0a19a8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:26.728169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cfeddf45-45bb-4458-837c-7e509f0a19a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:26.772394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.781494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b14ae-9f18-4cdd-b92e-1a64dee482bb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.809568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 404/804] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:26.825688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.826394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.826671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.826968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.827671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.827988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.829380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.841546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 401/805] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.847378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeddf45-45bb-4458-837c-7e509f0a19a8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.948580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeddf45-45bb-4458-837c-7e509f0a19a8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.983469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.984019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.984294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.984560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.991894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.993347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:27.004803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:27.005749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:27.006723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 405/806] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:27.012118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.072761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.105275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.106568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.115491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6b3a0-eeaa-45fc-a69a-e528fbe156c9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.147530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.148314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.148588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.148855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.149572 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:27.149873 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.151403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.168546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 402/807] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.174514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.179415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a08297-a33f-4db9-ba9a-f05b2f004298 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.213646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd230bc-1f50-403c-8ae9-353fc9aae717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.214302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd230bc-1f50-403c-8ae9-353fc9aae717 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.214579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd230bc-1f50-403c-8ae9-353fc9aae717 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.214846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd230bc-1f50-403c-8ae9-353fc9aae717 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.224958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd230bc-1f50-403c-8ae9-353fc9aae717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3cd230bc-1f50-403c-8ae9-353fc9aae717 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:27.229810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:27.230771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:27.231818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 406/808] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:27.236987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd230bc-1f50-403c-8ae9-353fc9aae717 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.275557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecd271-78e1-467c-b0c9-fe5796494850 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.308919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.309547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.309819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.310106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.320253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.321779 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.323968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.333740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd230bc-1f50-403c-8ae9-353fc9aae717 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:27.337235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:27.338274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:27.339315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:27.339315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:27.339315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:27.339315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 403/809] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:27.344403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.347411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.369372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.369978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.370286 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.370562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.371211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:27.371516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.372876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.380910 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 407/810] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.386927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.447355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fc916-70c4-4167-af86-73ba8f15ea8d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.482330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.482938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.483212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.483480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.484119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:27.484421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.484513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93b154-8705-4d7c-82a3-98f34f36ea0a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.485790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.497397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 404/811] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:27.503064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.518108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc9c7c-6052-4053-99c9-af9b459ae198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.518742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc9c7c-6052-4053-99c9-af9b459ae198 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.519022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc9c7c-6052-4053-99c9-af9b459ae198 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.519295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc9c7c-6052-4053-99c9-af9b459ae198 None admin] PATH_INFO: `/v3/users/925cde0b33d24ca4804d26a9078fdf25` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.532509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc9c7c-6052-4053-99c9-af9b459ae198 None admin] RBAC: Authorizing `identity:get_user(user_id=925cde0b33d24ca4804d26a9078fdf25)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.534047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc9c7c-6052-4053-99c9-af9b459ae198 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.535735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 408/812] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/users/925cde0b33d24ca4804d26a9078fdf25 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.541596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc9c7c-6052-4053-99c9-af9b459ae198 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.553473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.563070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.604193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016321f-acf0-4965-ab21-a2f0aabb30ed None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.639761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.640393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.640671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.640941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.642979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc9c7c-6052-4053-99c9-af9b459ae198 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.647605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.649124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:27.660828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:27.661866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:27.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 405/813] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:27.668073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.684662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8df8d-2a72-4cf1-8594-b004a16fac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.685293 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8df8d-2a72-4cf1-8594-b004a16fac13 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.685548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8df8d-2a72-4cf1-8594-b004a16fac13 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.685817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8df8d-2a72-4cf1-8594-b004a16fac13 None admin] PATH_INFO: `/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.686971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8df8d-2a72-4cf1-8594-b004a16fac13 None admin] RBAC: Authorizing `identity:get_project(project_id=c3105bdaf00b4f83b62b29c31ca57d12)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.688407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8df8d-2a72-4cf1-8594-b004a16fac13 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.690195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 409/814] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:27.696197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8df8d-2a72-4cf1-8594-b004a16fac13 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.769368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453dfa8-7b09-4725-a3d8-ca173891930b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.773348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.792182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.795855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8df8d-2a72-4cf1-8594-b004a16fac13 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.805291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.805893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.806197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.806465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.807149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:27.807445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.808853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.820370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 406/815] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:27.826737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.829031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37e916-219d-4ea9-94c8-5c325ce0dc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.829693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37e916-219d-4ea9-94c8-5c325ce0dc6f None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.829966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37e916-219d-4ea9-94c8-5c325ce0dc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.830277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37e916-219d-4ea9-94c8-5c325ce0dc6f None admin] PATH_INFO: `/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12/users/925cde0b33d24ca4804d26a9078fdf25/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.832227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37e916-219d-4ea9-94c8-5c325ce0dc6f None admin] RBAC: Authorizing `identity:create_grant(project_id=c3105bdaf00b4f83b62b29c31ca57d12, user_id=925cde0b33d24ca4804d26a9078fdf25, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.833644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37e916-219d-4ea9-94c8-5c325ce0dc6f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.847857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 410/816] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:27 2022] PUT /identity/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12/users/925cde0b33d24ca4804d26a9078fdf25/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:16:27.934218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c259989-bacf-4cea-ab1b-bfe539cb2c36 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.971451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e7edb-c19d-45be-b9a1-2c98aab32aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.972128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e7edb-c19d-45be-b9a1-2c98aab32aeb None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.972409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e7edb-c19d-45be-b9a1-2c98aab32aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.972680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e7edb-c19d-45be-b9a1-2c98aab32aeb None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/0722fc844689413796c658582ce00eb9/roles/f049cc06faaa41d99bb5f9f49c1d1096` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.975156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e7edb-c19d-45be-b9a1-2c98aab32aeb None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=0722fc844689413796c658582ce00eb9, role_id=f049cc06faaa41d99bb5f9f49c1d1096)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.976802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e7edb-c19d-45be-b9a1-2c98aab32aeb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.992583 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 407/817] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:27 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/0722fc844689413796c658582ce00eb9/roles/f049cc06faaa41d99bb5f9f49c1d1096 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:16:29.957681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe12fc5-7081-4cac-abba-7a334863b536 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.958289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe12fc5-7081-4cac-abba-7a334863b536 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.958289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe12fc5-7081-4cac-abba-7a334863b536 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.959622 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 411/818] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:29.965723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299ca0-1f1c-40ab-ae11-8d7a7c26afd9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.966163 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299ca0-1f1c-40ab-ae11-8d7a7c26afd9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.966434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299ca0-1f1c-40ab-ae11-8d7a7c26afd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.985540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-75299ca0-1f1c-40ab-ae11-8d7a7c26afd9 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:29.986766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-75299ca0-1f1c-40ab-ae11-8d7a7c26afd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:30.032381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.068060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 408/819] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:30.119890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab6dc4-4775-4714-943f-e009fe3f2dbf None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.120166 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab6dc4-4775-4714-943f-e009fe3f2dbf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.120432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab6dc4-4775-4714-943f-e009fe3f2dbf None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.121783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 412/820] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:30.127179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a96da-3822-4c3c-8712-7b39f3240124 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.127467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a96da-3822-4c3c-8712-7b39f3240124 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.127722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a96da-3822-4c3c-8712-7b39f3240124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.146371 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c46a96da-3822-4c3c-8712-7b39f3240124 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:30.147538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c46a96da-3822-4c3c-8712-7b39f3240124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:30.214436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 409/821] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:30.338945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab6dc4-4775-4714-943f-e009fe3f2dbf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.445215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab6dc4-4775-4714-943f-e009fe3f2dbf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.481194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18c1a5-dd1f-4a9e-8a14-3b0989452a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.481712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18c1a5-dd1f-4a9e-8a14-3b0989452a2c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.482011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18c1a5-dd1f-4a9e-8a14-3b0989452a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.482311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18c1a5-dd1f-4a9e-8a14-3b0989452a2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.483035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a96da-3822-4c3c-8712-7b39f3240124 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.489722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.494436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18c1a5-dd1f-4a9e-8a14-3b0989452a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9b18c1a5-dd1f-4a9e-8a14-3b0989452a2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:30.499727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:30.500702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:30.501587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 413/822] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:30.507059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18c1a5-dd1f-4a9e-8a14-3b0989452a2c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.585069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a96da-3822-4c3c-8712-7b39f3240124 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.587028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.608387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18c1a5-dd1f-4a9e-8a14-3b0989452a2c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.619986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea6ab5-83bf-45fb-9e03-8bdf316c7113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.620617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea6ab5-83bf-45fb-9e03-8bdf316c7113 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.620898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea6ab5-83bf-45fb-9e03-8bdf316c7113 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.621167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea6ab5-83bf-45fb-9e03-8bdf316c7113 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.631679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea6ab5-83bf-45fb-9e03-8bdf316c7113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b3ea6ab5-83bf-45fb-9e03-8bdf316c7113 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:30.636671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:30.637560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:30.638806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 410/823] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:30.643866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea6ab5-83bf-45fb-9e03-8bdf316c7113 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.645752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.646392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.646678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.646938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.647594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:30.647963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:30.649320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:30.657928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 414/824] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:30.663961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.720392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.744486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea6ab5-83bf-45fb-9e03-8bdf316c7113 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.761513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25390ae3-bc5a-4e73-82e5-90a8edc18070 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.777823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.778452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.778730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.778997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.779652 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:30.779951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:30.781322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:30.790107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 411/825] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:30.793802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc3d8c-4748-4532-bed5-48397ca6733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.794449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3d8c-4748-4532-bed5-48397ca6733c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.794730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3d8c-4748-4532-bed5-48397ca6733c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.794999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3d8c-4748-4532-bed5-48397ca6733c None admin] PATH_INFO: `/v3/users/925cde0b33d24ca4804d26a9078fdf25` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.795778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.796127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc3d8c-4748-4532-bed5-48397ca6733c None admin] RBAC: Authorizing `identity:get_user(user_id=925cde0b33d24ca4804d26a9078fdf25)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:30.797568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc3d8c-4748-4532-bed5-48397ca6733c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:30.802117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 415/826] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/users/925cde0b33d24ca4804d26a9078fdf25 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:30.804779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc3d8c-4748-4532-bed5-48397ca6733c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.820108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.895077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a7e1b-890f-49ce-9157-23829de02127 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.906541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc3d8c-4748-4532-bed5-48397ca6733c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.930035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.930701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.930967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.931217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.933410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.941859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea4730-bd56-4994-bde6-e885bed0d31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.942034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:30.942489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea4730-bd56-4994-bde6-e885bed0d31b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.942765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea4730-bd56-4994-bde6-e885bed0d31b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.943035 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea4730-bd56-4994-bde6-e885bed0d31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.943363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:30.957544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:30.958596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:30.959473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:30.959473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:30.959473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:30.959473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 412/827] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:30.964899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.014345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea4730-bd56-4994-bde6-e885bed0d31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.015697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea4730-bd56-4994-bde6-e885bed0d31b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.041022 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.065186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723b973-4482-47c8-b158-7bd2f320da9c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.098611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.099250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.099496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.099766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.100408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:31.100710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.102187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.113800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 413/828] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:31.114475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 416/829] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 07:16:31.119669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.120423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea4730-bd56-4994-bde6-e885bed0d31b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.147080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.219249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea4730-bd56-4994-bde6-e885bed0d31b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.220738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fcf23-0c82-467f-bc2d-7b33c5253a59 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.253343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072edf4-a973-4a8c-9f11-20ab2af8df4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.253963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072edf4-a973-4a8c-9f11-20ab2af8df4c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.254272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072edf4-a973-4a8c-9f11-20ab2af8df4c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.254538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072edf4-a973-4a8c-9f11-20ab2af8df4c None admin] PATH_INFO: `/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.255029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.255645 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072edf4-a973-4a8c-9f11-20ab2af8df4c None admin] RBAC: Authorizing `identity:get_project(project_id=c3105bdaf00b4f83b62b29c31ca57d12)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.255736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.255930 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.256207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.257067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072edf4-a973-4a8c-9f11-20ab2af8df4c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.258872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 417/830] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:31.259508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.264473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.264989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072edf4-a973-4a8c-9f11-20ab2af8df4c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.265998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:31.278049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:31.279190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:31.280725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 414/831] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:31.284825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.359996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.367086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072edf4-a973-4a8c-9f11-20ab2af8df4c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.388668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb41c8-fcac-47e9-af7d-7d57c81d85b7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.405963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.406673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.406952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.407222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.408007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=c3105bdaf00b4f83b62b29c31ca57d12, user.id=925cde0b33d24ca4804d26a9078fdf25) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:31.408324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.409790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.416295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 418/832] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/role_assignments?user.id=925cde0b33d24ca4804d26a9078fdf25&scope.project.id=c3105bdaf00b4f83b62b29c31ca57d12&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:31.426617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.427243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.427515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.427780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.429146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:31.429146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.430766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.451531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 415/833] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:31.457589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.568467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a0348-52a1-4787-8df4-5a00581789a8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.610447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4f63b-7d6e-46a8-acda-c8f06ea06e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.611027 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f63b-7d6e-46a8-acda-c8f06ea06e9c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.611295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f63b-7d6e-46a8-acda-c8f06ea06e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.611576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f63b-7d6e-46a8-acda-c8f06ea06e9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.641690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.685057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4f63b-7d6e-46a8-acda-c8f06ea06e9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.686552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4f63b-7d6e-46a8-acda-c8f06ea06e9c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.789712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 419/834] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:31.795971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.852386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.902541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58f0d-655d-40a7-bc66-2135a4583848 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.940850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.941483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.941765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.942035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.952477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.953920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:31.969787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:31.970896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:31.971777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:31.971777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:31.971777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:31.971777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 416/835] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:31.977145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4f63b-7d6e-46a8-acda-c8f06ea06e9c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.039001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.081351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4f63b-7d6e-46a8-acda-c8f06ea06e9c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.116986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.117770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.118048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.118344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.118983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:32.119334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.120731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.133118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 420/836] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:32.138960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.244609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10441a-f3e9-4c36-8deb-d4e9e4e7cf75 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.251504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.278335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.278957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.279237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.279512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.286153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.287590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:32.299384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:32.300277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:32.301159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 417/837] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:32.306902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.410666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f494f6e-3290-4af4-b25a-41f16220b93d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.430501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.446366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca3290-a053-44b5-b2a7-3f83b28d2b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.446971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3290-a053-44b5-b2a7-3f83b28d2b77 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.447241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3290-a053-44b5-b2a7-3f83b28d2b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.447507 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3290-a053-44b5-b2a7-3f83b28d2b77 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.448195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca3290-a053-44b5-b2a7-3f83b28d2b77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:32.448499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca3290-a053-44b5-b2a7-3f83b28d2b77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.449958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca3290-a053-44b5-b2a7-3f83b28d2b77 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.462761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 421/838] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:32.469056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.571750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd1476-7c18-4c96-b094-32b9bc169b21 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.606774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70321d9a-76c9-4110-8a57-a1a3c428d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.607379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321d9a-76c9-4110-8a57-a1a3c428d677 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.607654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321d9a-76c9-4110-8a57-a1a3c428d677 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.607924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321d9a-76c9-4110-8a57-a1a3c428d677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.608676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321d9a-76c9-4110-8a57-a1a3c428d677 None admin] RBAC: Adding query filter params (role.id=f049cc06faaa41d99bb5f9f49c1d1096, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=0722fc844689413796c658582ce00eb9) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:32.608978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321d9a-76c9-4110-8a57-a1a3c428d677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.610450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321d9a-76c9-4110-8a57-a1a3c428d677 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.616756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 418/839] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/role_assignments?user.id=0722fc844689413796c658582ce00eb9&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=f049cc06faaa41d99bb5f9f49c1d1096 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:33.573515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8c772-e108-413a-8d1a-9150d1faaeaa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.574475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8c772-e108-413a-8d1a-9150d1faaeaa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.574475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8c772-e108-413a-8d1a-9150d1faaeaa None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.576574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 422/840] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:33.582939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6dfca-a8ac-4bad-8991-f685e3bfa52b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.583374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6dfca-a8ac-4bad-8991-f685e3bfa52b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.583509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6dfca-a8ac-4bad-8991-f685e3bfa52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.610189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-88c6dfca-a8ac-4bad-8991-f685e3bfa52b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:33.611399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-88c6dfca-a8ac-4bad-8991-f685e3bfa52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:33.639595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:33.686590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 419/841] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:33.950700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8c772-e108-413a-8d1a-9150d1faaeaa None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.065307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8c772-e108-413a-8d1a-9150d1faaeaa None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.102564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f85e8-403a-443b-be7f-928c9ec09df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.103169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f85e8-403a-443b-be7f-928c9ec09df8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.103439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f85e8-403a-443b-be7f-928c9ec09df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.103699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f85e8-403a-443b-be7f-928c9ec09df8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.104826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f85e8-403a-443b-be7f-928c9ec09df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.106272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f85e8-403a-443b-be7f-928c9ec09df8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.107805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 423/842] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:34.113941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6dfca-a8ac-4bad-8991-f685e3bfa52b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.229827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6dfca-a8ac-4bad-8991-f685e3bfa52b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.265652 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d14a4-5a97-4797-b4df-e4300cd84d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.266267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d14a4-5a97-4797-b4df-e4300cd84d45 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.266539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d14a4-5a97-4797-b4df-e4300cd84d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.266805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d14a4-5a97-4797-b4df-e4300cd84d45 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.267449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d14a4-5a97-4797-b4df-e4300cd84d45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.268869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d14a4-5a97-4797-b4df-e4300cd84d45 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.292286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 420/843] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:34 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:36.445670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3abb91-ae26-4348-b2ef-0dc15fefd52f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.446263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3abb91-ae26-4348-b2ef-0dc15fefd52f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.446263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3abb91-ae26-4348-b2ef-0dc15fefd52f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.447587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 424/844] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:36.453869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4d63f-a4ab-431e-ac23-56167d3b69d3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.454632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4d63f-a4ab-431e-ac23-56167d3b69d3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.454809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4d63f-a4ab-431e-ac23-56167d3b69d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.476788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a0e4d63f-a4ab-431e-ac23-56167d3b69d3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:36.478610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a0e4d63f-a4ab-431e-ac23-56167d3b69d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:36.489304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.556410 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 421/845] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:36.829647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3abb91-ae26-4348-b2ef-0dc15fefd52f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.889278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.943677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3abb91-ae26-4348-b2ef-0dc15fefd52f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.979065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07697b-8c83-4755-a49e-a41c88ee2c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.979676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07697b-8c83-4755-a49e-a41c88ee2c3b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.979955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07697b-8c83-4755-a49e-a41c88ee2c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.980226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07697b-8c83-4755-a49e-a41c88ee2c3b None admin] PATH_INFO: `/v3/roles/5a2ee20f691f47539f8b1850459be9ca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.981638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07697b-8c83-4755-a49e-a41c88ee2c3b None admin] RBAC: Authorizing `identity:get_role(role_id=5a2ee20f691f47539f8b1850459be9ca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.983053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07697b-8c83-4755-a49e-a41c88ee2c3b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.984366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 425/846] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/roles/5a2ee20f691f47539f8b1850459be9ca => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:36.991069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4d63f-a4ab-431e-ac23-56167d3b69d3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.101181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4d63f-a4ab-431e-ac23-56167d3b69d3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.153113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624d87a-d29f-4255-89b8-01927820eb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.153738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d87a-d29f-4255-89b8-01927820eb9c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.154013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d87a-d29f-4255-89b8-01927820eb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.154313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d87a-d29f-4255-89b8-01927820eb9c None admin] PATH_INFO: `/v3/users/1a99e4486af24fc398292d2360f130dc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.169146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624d87a-d29f-4255-89b8-01927820eb9c None admin] RBAC: Authorizing `identity:get_user(user_id=1a99e4486af24fc398292d2360f130dc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.170642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624d87a-d29f-4255-89b8-01927820eb9c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.172252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 422/847] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/users/1a99e4486af24fc398292d2360f130dc => generated 286 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:37.180915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07697b-8c83-4755-a49e-a41c88ee2c3b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.287094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07697b-8c83-4755-a49e-a41c88ee2c3b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.303617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:37.323542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa250263-536c-40bb-a080-57970665eb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.324150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa250263-536c-40bb-a080-57970665eb8c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.324423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa250263-536c-40bb-a080-57970665eb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.324690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa250263-536c-40bb-a080-57970665eb8c None admin] PATH_INFO: `/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.325830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa250263-536c-40bb-a080-57970665eb8c None admin] RBAC: Authorizing `identity:get_project(project_id=c3105bdaf00b4f83b62b29c31ca57d12)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.327289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa250263-536c-40bb-a080-57970665eb8c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.328878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 426/848] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:37 2022] GET /identity/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:37.335366 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624d87a-d29f-4255-89b8-01927820eb9c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.339781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:37.445148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624d87a-d29f-4255-89b8-01927820eb9c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.479920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32899de-e9b2-4b58-9307-873b700dea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.480526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32899de-e9b2-4b58-9307-873b700dea28 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.480812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32899de-e9b2-4b58-9307-873b700dea28 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.481076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32899de-e9b2-4b58-9307-873b700dea28 None admin] PATH_INFO: `/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12/users/1a99e4486af24fc398292d2360f130dc/roles/5a2ee20f691f47539f8b1850459be9ca` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.483359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32899de-e9b2-4b58-9307-873b700dea28 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3105bdaf00b4f83b62b29c31ca57d12, user_id=1a99e4486af24fc398292d2360f130dc, role_id=5a2ee20f691f47539f8b1850459be9ca)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.484776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32899de-e9b2-4b58-9307-873b700dea28 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.500708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 423/849] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:37 2022] PUT /identity/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12/users/1a99e4486af24fc398292d2360f130dc/roles/5a2ee20f691f47539f8b1850459be9ca => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:39.593515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fea61b-d496-4217-a06e-69ceab69cb6a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.594420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fea61b-d496-4217-a06e-69ceab69cb6a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.594420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fea61b-d496-4217-a06e-69ceab69cb6a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.596441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 427/850] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:39.602839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5373c-0a2e-4be4-8a68-f8ba296c0ceb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.603279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5373c-0a2e-4be4-8a68-f8ba296c0ceb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.603388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5373c-0a2e-4be4-8a68-f8ba296c0ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.634375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-60d5373c-0a2e-4be4-8a68-f8ba296c0ceb None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:39.636071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-60d5373c-0a2e-4be4-8a68-f8ba296c0ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:39.710885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.726869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 424/851] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:39.991845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fea61b-d496-4217-a06e-69ceab69cb6a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:40.111401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fea61b-d496-4217-a06e-69ceab69cb6a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.147147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f51b95-9a2a-4464-a325-ace4afe39202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.147846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f51b95-9a2a-4464-a325-ace4afe39202 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.148115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f51b95-9a2a-4464-a325-ace4afe39202 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.148419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f51b95-9a2a-4464-a325-ace4afe39202 None admin] PATH_INFO: `/v3/roles/5a2ee20f691f47539f8b1850459be9ca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.149931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f51b95-9a2a-4464-a325-ace4afe39202 None admin] RBAC: Authorizing `identity:get_role(role_id=5a2ee20f691f47539f8b1850459be9ca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.151341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f51b95-9a2a-4464-a325-ace4afe39202 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.152676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 428/852] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/roles/5a2ee20f691f47539f8b1850459be9ca => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:40.159664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5373c-0a2e-4be4-8a68-f8ba296c0ceb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:40.267931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5373c-0a2e-4be4-8a68-f8ba296c0ceb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.303597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a32b-9b53-4bad-9578-2f20b95035b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.304297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a32b-9b53-4bad-9578-2f20b95035b4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.304646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a32b-9b53-4bad-9578-2f20b95035b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.304914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a32b-9b53-4bad-9578-2f20b95035b4 None admin] PATH_INFO: `/v3/users/1a99e4486af24fc398292d2360f130dc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.306149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51a32b-9b53-4bad-9578-2f20b95035b4 None admin] RBAC: Authorizing `identity:get_user(user_id=1a99e4486af24fc398292d2360f130dc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.307594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51a32b-9b53-4bad-9578-2f20b95035b4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.309127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 425/853] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:40 2022] GET /identity/v3/users/1a99e4486af24fc398292d2360f130dc => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:40.315311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f51b95-9a2a-4464-a325-ace4afe39202 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:40.373236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:40.424445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f51b95-9a2a-4464-a325-ace4afe39202 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.458871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7896f6-9e03-4cbc-9bae-aa6110f6d2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.459470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7896f6-9e03-4cbc-9bae-aa6110f6d2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.459751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7896f6-9e03-4cbc-9bae-aa6110f6d2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.460043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7896f6-9e03-4cbc-9bae-aa6110f6d2c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.530708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7896f6-9e03-4cbc-9bae-aa6110f6d2c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.532088 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7896f6-9e03-4cbc-9bae-aa6110f6d2c8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.587384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:40.645958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 429/854] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 07:16:40.652363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a32b-9b53-4bad-9578-2f20b95035b4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:40.725980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:40.779324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a32b-9b53-4bad-9578-2f20b95035b4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.824035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a88d6-8113-407b-a439-99820c253404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.824587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a88d6-8113-407b-a439-99820c253404 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.824861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a88d6-8113-407b-a439-99820c253404 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.825130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a88d6-8113-407b-a439-99820c253404 None admin] PATH_INFO: `/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.826279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a88d6-8113-407b-a439-99820c253404 None admin] RBAC: Authorizing `identity:get_project(project_id=c3105bdaf00b4f83b62b29c31ca57d12)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.827701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a88d6-8113-407b-a439-99820c253404 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.829344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 426/855] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:40 2022] GET /identity/v3/projects/c3105bdaf00b4f83b62b29c31ca57d12 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:40.835962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7896f6-9e03-4cbc-9bae-aa6110f6d2c8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:40.949730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7896f6-9e03-4cbc-9bae-aa6110f6d2c8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:40.986350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54d00c-4b2f-41dd-ab2e-21b276c74eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:40.986953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54d00c-4b2f-41dd-ab2e-21b276c74eac None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:40.987216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54d00c-4b2f-41dd-ab2e-21b276c74eac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:40.987485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54d00c-4b2f-41dd-ab2e-21b276c74eac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:40.988203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54d00c-4b2f-41dd-ab2e-21b276c74eac None admin] RBAC: Adding query filter params (role.id=5a2ee20f691f47539f8b1850459be9ca, scope.project.id=c3105bdaf00b4f83b62b29c31ca57d12, user.id=1a99e4486af24fc398292d2360f130dc) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:40.988508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54d00c-4b2f-41dd-ab2e-21b276c74eac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:40.989943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54d00c-4b2f-41dd-ab2e-21b276c74eac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:40.996646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 430/856] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:40 2022] GET /identity/v3/role_assignments?user.id=1a99e4486af24fc398292d2360f130dc&scope.project.id=c3105bdaf00b4f83b62b29c31ca57d12&role.id=5a2ee20f691f47539f8b1850459be9ca => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:43.142895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907484f4-45f1-4d5c-9ba2-4ec800e11ad9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:43.143444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907484f4-45f1-4d5c-9ba2-4ec800e11ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:43.143444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907484f4-45f1-4d5c-9ba2-4ec800e11ad9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:43.145498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 427/857] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:43.152146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b365310-3389-4bc4-96ae-e437ec416083 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:43.152527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b365310-3389-4bc4-96ae-e437ec416083 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:43.152698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b365310-3389-4bc4-96ae-e437ec416083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:43.171404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4b365310-3389-4bc4-96ae-e437ec416083 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:43.172543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4b365310-3389-4bc4-96ae-e437ec416083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:43.183614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:43.242819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 431/858] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:43.510494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907484f4-45f1-4d5c-9ba2-4ec800e11ad9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:43.627433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907484f4-45f1-4d5c-9ba2-4ec800e11ad9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:43.655900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:43.663109 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05308541-1eba-4aa7-b30f-1f3c43ad7eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:43.663765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05308541-1eba-4aa7-b30f-1f3c43ad7eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:43.664045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05308541-1eba-4aa7-b30f-1f3c43ad7eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:43.664319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05308541-1eba-4aa7-b30f-1f3c43ad7eb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:43.666796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05308541-1eba-4aa7-b30f-1f3c43ad7eb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:43.670832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05308541-1eba-4aa7-b30f-1f3c43ad7eb4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:43.672510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 428/859] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:43.678973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b365310-3389-4bc4-96ae-e437ec416083 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:43.799185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b365310-3389-4bc4-96ae-e437ec416083 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:43.834080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe4e27-914d-47bd-8649-8c5f237317bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:43.834707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe4e27-914d-47bd-8649-8c5f237317bf None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:43.835085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe4e27-914d-47bd-8649-8c5f237317bf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:43.835575 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe4e27-914d-47bd-8649-8c5f237317bf None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:43.836628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe4e27-914d-47bd-8649-8c5f237317bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:43.838125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe4e27-914d-47bd-8649-8c5f237317bf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:43.861130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:43.869597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 432/860] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:43 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:46.036192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620975f8-3b77-40ab-a02e-5fc30f17bdc8 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620975f8-3b77-40ab-a02e-5fc30f17bdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.037133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620975f8-3b77-40ab-a02e-5fc30f17bdc8 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.039280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 429/861] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:46.045738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8bcc-4f03-4ec4-8487-2bba945d031b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.046223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8bcc-4f03-4ec4-8487-2bba945d031b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.046322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8bcc-4f03-4ec4-8487-2bba945d031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.068230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-13cd8bcc-4f03-4ec4-8487-2bba945d031b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:46.069518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-13cd8bcc-4f03-4ec4-8487-2bba945d031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:46.138674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 433/862] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:46.407776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620975f8-3b77-40ab-a02e-5fc30f17bdc8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:46.523397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620975f8-3b77-40ab-a02e-5fc30f17bdc8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:46.557983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3e679-9add-4d07-a8c2-4863fde7541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:46.558624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3e679-9add-4d07-a8c2-4863fde7541c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.558909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3e679-9add-4d07-a8c2-4863fde7541c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.559175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3e679-9add-4d07-a8c2-4863fde7541c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.560380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3e679-9add-4d07-a8c2-4863fde7541c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:46.561802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3e679-9add-4d07-a8c2-4863fde7541c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:46.563368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 430/863] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:46.572161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8bcc-4f03-4ec4-8487-2bba945d031b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:46.684802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8bcc-4f03-4ec4-8487-2bba945d031b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:46.711894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:46.720922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2f3d3-a687-4e69-8470-735f95289a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:46.721505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f3d3-a687-4e69-8470-735f95289a65 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.721777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f3d3-a687-4e69-8470-735f95289a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.722039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f3d3-a687-4e69-8470-735f95289a65 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.722707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2f3d3-a687-4e69-8470-735f95289a65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:46.724149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2f3d3-a687-4e69-8470-735f95289a65 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:46.748414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 434/864] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:46 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:48.903895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bc9e5-3b37-41ce-b658-51df03c0a5a7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.904789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bc9e5-3b37-41ce-b658-51df03c0a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.904789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bc9e5-3b37-41ce-b658-51df03c0a5a7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.906970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 431/865] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:48.913261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3b7f8-93ef-4ec1-a940-d83eb1d6a276 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.913987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3b7f8-93ef-4ec1-a940-d83eb1d6a276 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.914175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3b7f8-93ef-4ec1-a940-d83eb1d6a276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.938515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e2e3b7f8-93ef-4ec1-a940-d83eb1d6a276 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:48.939694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e2e3b7f8-93ef-4ec1-a940-d83eb1d6a276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:49.012936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 435/866] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:49.282223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bc9e5-3b37-41ce-b658-51df03c0a5a7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.352932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.392570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bc9e5-3b37-41ce-b658-51df03c0a5a7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.428218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeb54b-d96e-4738-a81f-4b3b9260da26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.428860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb54b-d96e-4738-a81f-4b3b9260da26 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.429135 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb54b-d96e-4738-a81f-4b3b9260da26 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.429406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeb54b-d96e-4738-a81f-4b3b9260da26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.440908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdeb54b-d96e-4738-a81f-4b3b9260da26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bcdeb54b-d96e-4738-a81f-4b3b9260da26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:49.445760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:49.446830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:16:49.447763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 432/867] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:49.455713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3b7f8-93ef-4ec1-a940-d83eb1d6a276 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.574060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3b7f8-93ef-4ec1-a940-d83eb1d6a276 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.588488 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.610997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.611594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.611874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.612140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.612795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:49.613088 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.614493 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.622383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 436/868] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:49.628389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeb54b-d96e-4738-a81f-4b3b9260da26 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.739797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeb54b-d96e-4738-a81f-4b3b9260da26 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.767913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.778669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880dd8a-bd8f-4c2a-bf5e-78102df67617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.779300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880dd8a-bd8f-4c2a-bf5e-78102df67617 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.779575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880dd8a-bd8f-4c2a-bf5e-78102df67617 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.779838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880dd8a-bd8f-4c2a-bf5e-78102df67617 None admin] PATH_INFO: `/v3/users/5edf9c4d09af43feb48a77f946264135` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.796495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880dd8a-bd8f-4c2a-bf5e-78102df67617 None admin] RBAC: Authorizing `identity:get_user(user_id=5edf9c4d09af43feb48a77f946264135)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.798017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880dd8a-bd8f-4c2a-bf5e-78102df67617 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.799637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 433/869] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/users/5edf9c4d09af43feb48a77f946264135 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:49.806619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.919421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01bd0c-954e-49a5-9901-a3728098e131 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.956677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41097b96-c590-4346-a261-d4c46c5bdfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.957289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41097b96-c590-4346-a261-d4c46c5bdfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.957561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41097b96-c590-4346-a261-d4c46c5bdfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.957795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41097b96-c590-4346-a261-d4c46c5bdfb4 None admin] PATH_INFO: `/v3/projects/c9dedcbb5a1f4001b983b9336908064f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.958993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41097b96-c590-4346-a261-d4c46c5bdfb4 None admin] RBAC: Authorizing `identity:get_project(project_id=c9dedcbb5a1f4001b983b9336908064f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.960743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41097b96-c590-4346-a261-d4c46c5bdfb4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.962745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 437/870] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/projects/c9dedcbb5a1f4001b983b9336908064f => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:49.968879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880dd8a-bd8f-4c2a-bf5e-78102df67617 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:50.202886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880dd8a-bd8f-4c2a-bf5e-78102df67617 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:50.245308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36914e5c-a5e0-45d0-a1df-75aa2bad7535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:50.246815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36914e5c-a5e0-45d0-a1df-75aa2bad7535 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:50.247296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36914e5c-a5e0-45d0-a1df-75aa2bad7535 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:50.248310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36914e5c-a5e0-45d0-a1df-75aa2bad7535 None admin] PATH_INFO: `/v3/projects/c9dedcbb5a1f4001b983b9336908064f/users/5edf9c4d09af43feb48a77f946264135/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:50.251925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36914e5c-a5e0-45d0-a1df-75aa2bad7535 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9dedcbb5a1f4001b983b9336908064f, user_id=5edf9c4d09af43feb48a77f946264135, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:50.254329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36914e5c-a5e0-45d0-a1df-75aa2bad7535 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:50.275552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 434/871] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:49 2022] PUT /identity/v3/projects/c9dedcbb5a1f4001b983b9336908064f/users/5edf9c4d09af43feb48a77f946264135/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:52.369103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8707c0-b7ba-4d29-a841-39218a5bfa21 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.369954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8707c0-b7ba-4d29-a841-39218a5bfa21 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.369954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8707c0-b7ba-4d29-a841-39218a5bfa21 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.372114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 438/872] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:52.378439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9dd6f-32b6-490e-9c87-d0bb3f301040 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.378912 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9dd6f-32b6-490e-9c87-d0bb3f301040 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.379001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9dd6f-32b6-490e-9c87-d0bb3f301040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.399302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8dd9dd6f-32b6-490e-9c87-d0bb3f301040 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:52.400460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8dd9dd6f-32b6-490e-9c87-d0bb3f301040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:52.441511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:52.470137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 435/873] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:52.736676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8707c0-b7ba-4d29-a841-39218a5bfa21 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.792211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:52.855811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8707c0-b7ba-4d29-a841-39218a5bfa21 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.899906 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffa753-7afa-4ae7-a956-f1414cb44783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.900537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffa753-7afa-4ae7-a956-f1414cb44783 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.900816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffa753-7afa-4ae7-a956-f1414cb44783 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.901091 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffa753-7afa-4ae7-a956-f1414cb44783 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.912567 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffa753-7afa-4ae7-a956-f1414cb44783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-91ffa753-7afa-4ae7-a956-f1414cb44783 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:52.917318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:52.918424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:16:52.919408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 439/874] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:52.924760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9dd6f-32b6-490e-9c87-d0bb3f301040 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.024537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9dd6f-32b6-490e-9c87-d0bb3f301040 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.059132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.059734 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.060014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.060282 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.060938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:53.061245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.062646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.071080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 436/875] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:53.077177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffa753-7afa-4ae7-a956-f1414cb44783 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.191530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffa753-7afa-4ae7-a956-f1414cb44783 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.195074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:53.228949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2132ec9-33c7-4403-ab4c-6e71bb0a72ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.229550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2132ec9-33c7-4403-ab4c-6e71bb0a72ca None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.229822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2132ec9-33c7-4403-ab4c-6e71bb0a72ca None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.230085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2132ec9-33c7-4403-ab4c-6e71bb0a72ca None admin] PATH_INFO: `/v3/users/5edf9c4d09af43feb48a77f946264135` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.231657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2132ec9-33c7-4403-ab4c-6e71bb0a72ca None admin] RBAC: Authorizing `identity:get_user(user_id=5edf9c4d09af43feb48a77f946264135)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.233696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2132ec9-33c7-4403-ab4c-6e71bb0a72ca None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.235283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 440/876] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/users/5edf9c4d09af43feb48a77f946264135 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:53.241297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.277253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:53.348013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c823378-8f5a-4a97-b0b5-01eb1a2841cc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.384660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254034c9-e23e-4d28-a0dc-febf390a3dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.385293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254034c9-e23e-4d28-a0dc-febf390a3dca None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.385498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254034c9-e23e-4d28-a0dc-febf390a3dca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.385760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254034c9-e23e-4d28-a0dc-febf390a3dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.460106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254034c9-e23e-4d28-a0dc-febf390a3dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.462623 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254034c9-e23e-4d28-a0dc-febf390a3dca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.498581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:53.582051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 437/877] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:53.589522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2132ec9-33c7-4403-ab4c-6e71bb0a72ca None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.711731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2132ec9-33c7-4403-ab4c-6e71bb0a72ca None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.757891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcea29-a514-45fa-990b-e3be1b93e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.758575 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcea29-a514-45fa-990b-e3be1b93e68f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.758855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcea29-a514-45fa-990b-e3be1b93e68f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.759126 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcea29-a514-45fa-990b-e3be1b93e68f None admin] PATH_INFO: `/v3/projects/c9dedcbb5a1f4001b983b9336908064f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.760309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fcea29-a514-45fa-990b-e3be1b93e68f None admin] RBAC: Authorizing `identity:get_project(project_id=c9dedcbb5a1f4001b983b9336908064f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.761754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fcea29-a514-45fa-990b-e3be1b93e68f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.763466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 441/878] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/projects/c9dedcbb5a1f4001b983b9336908064f => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:53.770004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254034c9-e23e-4d28-a0dc-febf390a3dca None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:53.873971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254034c9-e23e-4d28-a0dc-febf390a3dca None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:53.910040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02945ea1-7dcc-4434-ac5a-68834bbd5764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:53.910659 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02945ea1-7dcc-4434-ac5a-68834bbd5764 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:53.910928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02945ea1-7dcc-4434-ac5a-68834bbd5764 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:53.911191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02945ea1-7dcc-4434-ac5a-68834bbd5764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:53.911894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02945ea1-7dcc-4434-ac5a-68834bbd5764 None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=c9dedcbb5a1f4001b983b9336908064f, user.id=5edf9c4d09af43feb48a77f946264135) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:53.912193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02945ea1-7dcc-4434-ac5a-68834bbd5764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:53.913603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02945ea1-7dcc-4434-ac5a-68834bbd5764 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:53.916848 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:53.920169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 438/879] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:53 2022] GET /identity/v3/role_assignments?user.id=5edf9c4d09af43feb48a77f946264135&scope.project.id=c9dedcbb5a1f4001b983b9336908064f&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:56.053451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2eee-c702-49ee-8338-bf166c0e2632 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:56.054369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2eee-c702-49ee-8338-bf166c0e2632 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:56.054369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2eee-c702-49ee-8338-bf166c0e2632 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:56.056609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 442/880] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:56.065439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036037db-8a40-4d28-b20e-cb1ade70a629 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:56.065854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036037db-8a40-4d28-b20e-cb1ade70a629 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:56.066016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036037db-8a40-4d28-b20e-cb1ade70a629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:56.087981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-036037db-8a40-4d28-b20e-cb1ade70a629 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:56.089164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-036037db-8a40-4d28-b20e-cb1ade70a629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:56.161892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 439/881] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:56.428060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2eee-c702-49ee-8338-bf166c0e2632 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:56.476439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:56.538290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2eee-c702-49ee-8338-bf166c0e2632 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:56.573786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c55aa1-3e3e-4d9a-baa6-ea16aecebc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:56.574409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55aa1-3e3e-4d9a-baa6-ea16aecebc2e None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:56.574686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55aa1-3e3e-4d9a-baa6-ea16aecebc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:56.574957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55aa1-3e3e-4d9a-baa6-ea16aecebc2e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:56.575624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c55aa1-3e3e-4d9a-baa6-ea16aecebc2e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:56.576997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c55aa1-3e3e-4d9a-baa6-ea16aecebc2e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:56.596173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 443/882] 10.222.0.25 () {60 vars in 1138 bytes} [Wed Nov 2 07:16:56 2022] POST /identity/v3/domains => generated 267 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:58.726017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dbf2a-787c-4bb4-943f-0b7c70e099f9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.726687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dbf2a-787c-4bb4-943f-0b7c70e099f9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.726687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dbf2a-787c-4bb4-943f-0b7c70e099f9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.728045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 440/883] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:58.734641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7ae8-516c-4c19-8f0b-9fe69e177ac2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.735364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7ae8-516c-4c19-8f0b-9fe69e177ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.735516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e7ae8-516c-4c19-8f0b-9fe69e177ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.759326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ae4e7ae8-516c-4c19-8f0b-9fe69e177ac2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:58.760515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ae4e7ae8-516c-4c19-8f0b-9fe69e177ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:58.840959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 444/884] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:59.103211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dbf2a-787c-4bb4-943f-0b7c70e099f9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.210840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dbf2a-787c-4bb4-943f-0b7c70e099f9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.244756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.252065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aa8ac-af26-4daa-87e2-4207d1715d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.252692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa8ac-af26-4daa-87e2-4207d1715d6e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.252966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa8ac-af26-4daa-87e2-4207d1715d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.253233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa8ac-af26-4daa-87e2-4207d1715d6e None admin] PATH_INFO: `/v3/domains/cc0f0ee99edc41c9885b6b57342fee7a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.265602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495aa8ac-af26-4daa-87e2-4207d1715d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=cc0f0ee99edc41c9885b6b57342fee7a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.267094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495aa8ac-af26-4daa-87e2-4207d1715d6e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.268688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 441/885] 10.222.0.25 () {56 vars in 1183 bytes} [Wed Nov 2 07:16:59 2022] GET /identity/v3/domains/cc0f0ee99edc41c9885b6b57342fee7a => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:59.274996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e7ae8-516c-4c19-8f0b-9fe69e177ac2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:59.307220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:59.384756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e7ae8-516c-4c19-8f0b-9fe69e177ac2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:59.425967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e66d9-6727-4386-a58c-af7cac452bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:59.426593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e66d9-6727-4386-a58c-af7cac452bf6 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:59.426872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e66d9-6727-4386-a58c-af7cac452bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:59.427143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e66d9-6727-4386-a58c-af7cac452bf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:59.427790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e66d9-6727-4386-a58c-af7cac452bf6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:59.429327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e66d9-6727-4386-a58c-af7cac452bf6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:59.461909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 445/886] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:59 2022] POST /identity/v3/projects => generated 361 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:01.623315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594ceb-ee19-4015-b830-7d43cb4b5062 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:01.624211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594ceb-ee19-4015-b830-7d43cb4b5062 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:01.624211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594ceb-ee19-4015-b830-7d43cb4b5062 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:01.626349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 442/887] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:01.631883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c29d9-5d67-465a-8bbd-f1e2d8235648 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:01.632608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c29d9-5d67-465a-8bbd-f1e2d8235648 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:01.632608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c29d9-5d67-465a-8bbd-f1e2d8235648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:01.652039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5c4c29d9-5d67-465a-8bbd-f1e2d8235648 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:01.653215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c4c29d9-5d67-465a-8bbd-f1e2d8235648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:01.721727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 446/888] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:01.983738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594ceb-ee19-4015-b830-7d43cb4b5062 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:02.104985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594ceb-ee19-4015-b830-7d43cb4b5062 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:02.140774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643100c-2342-4fed-9759-0079a6680434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:02.141403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643100c-2342-4fed-9759-0079a6680434 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:02.141677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643100c-2342-4fed-9759-0079a6680434 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:02.141940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643100c-2342-4fed-9759-0079a6680434 None admin] PATH_INFO: `/v3/domains/cc0f0ee99edc41c9885b6b57342fee7a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:02.143201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643100c-2342-4fed-9759-0079a6680434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cc0f0ee99edc41c9885b6b57342fee7a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:02.144632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643100c-2342-4fed-9759-0079a6680434 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:02.146153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 443/889] 10.222.0.25 () {56 vars in 1183 bytes} [Wed Nov 2 07:17:01 2022] GET /identity/v3/domains/cc0f0ee99edc41c9885b6b57342fee7a => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:02.152535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c29d9-5d67-465a-8bbd-f1e2d8235648 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:02.171454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:02.263470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c29d9-5d67-465a-8bbd-f1e2d8235648 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:02.300095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708b3e2-a113-43e5-8772-74d48f3c8c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:02.300690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708b3e2-a113-43e5-8772-74d48f3c8c90 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:02.300958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708b3e2-a113-43e5-8772-74d48f3c8c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:02.301227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708b3e2-a113-43e5-8772-74d48f3c8c90 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:02.301869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708b3e2-a113-43e5-8772-74d48f3c8c90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:02.303337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708b3e2-a113-43e5-8772-74d48f3c8c90 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:02.327801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 447/890] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:02 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:04.484026 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff4e2-7704-4046-813b-f9883771d938 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.485036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff4e2-7704-4046-813b-f9883771d938 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.485036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff4e2-7704-4046-813b-f9883771d938 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.487150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 444/891] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:17:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:04.493075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33af93e-da7f-47b2-9f46-a2f1fbeaa327 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.493542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33af93e-da7f-47b2-9f46-a2f1fbeaa327 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.493632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33af93e-da7f-47b2-9f46-a2f1fbeaa327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.513336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a33af93e-da7f-47b2-9f46-a2f1fbeaa327 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:04.514544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a33af93e-da7f-47b2-9f46-a2f1fbeaa327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:04.579469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.586019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 448/892] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:04.849131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcff4e2-7704-4046-813b-f9883771d938 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.917603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.966131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcff4e2-7704-4046-813b-f9883771d938 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.000184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a1a24-5088-4c07-8c22-b4c80fdd2961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.000801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a1a24-5088-4c07-8c22-b4c80fdd2961 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.001081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a1a24-5088-4c07-8c22-b4c80fdd2961 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.001347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a1a24-5088-4c07-8c22-b4c80fdd2961 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.012351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a1a24-5088-4c07-8c22-b4c80fdd2961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-912a1a24-5088-4c07-8c22-b4c80fdd2961 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:05.017093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:05.018280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:17:05.019503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 445/893] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:05.026872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33af93e-da7f-47b2-9f46-a2f1fbeaa327 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.135310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33af93e-da7f-47b2-9f46-a2f1fbeaa327 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.171124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.171715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.171990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.172260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.172909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:05.173207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.174600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.183372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 449/894] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:05.189505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a1a24-5088-4c07-8c22-b4c80fdd2961 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.302468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a1a24-5088-4c07-8c22-b4c80fdd2961 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.319645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:05.338562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3c3bd-ccc3-4a73-9d2b-891d2432b2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.339180 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3c3bd-ccc3-4a73-9d2b-891d2432b2af None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.339457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3c3bd-ccc3-4a73-9d2b-891d2432b2af None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.339733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3c3bd-ccc3-4a73-9d2b-891d2432b2af None admin] PATH_INFO: `/v3/users/8b0ec6ad218d4286a5d0d0845262fad0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.352687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3c3bd-ccc3-4a73-9d2b-891d2432b2af None admin] RBAC: Authorizing `identity:get_user(user_id=8b0ec6ad218d4286a5d0d0845262fad0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.354158 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3c3bd-ccc3-4a73-9d2b-891d2432b2af None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.355745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 446/895] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:17:05 2022] GET /identity/v3/users/8b0ec6ad218d4286a5d0d0845262fad0 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:05.361825 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.425635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:05.472860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a22964-197c-4672-9f3c-2c379f5bdf71 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.507729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b0cee-8912-4c38-94c2-87f257dcc236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.508394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0cee-8912-4c38-94c2-87f257dcc236 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.508621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0cee-8912-4c38-94c2-87f257dcc236 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.508891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0cee-8912-4c38-94c2-87f257dcc236 None admin] PATH_INFO: `/v3/projects/f0ce673894c2474fbaccb6becda86aa5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.510063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b0cee-8912-4c38-94c2-87f257dcc236 None admin] RBAC: Authorizing `identity:get_project(project_id=f0ce673894c2474fbaccb6becda86aa5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.511534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b0cee-8912-4c38-94c2-87f257dcc236 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.513156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 450/896] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:17:05 2022] GET /identity/v3/projects/f0ce673894c2474fbaccb6becda86aa5 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:05.519685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3c3bd-ccc3-4a73-9d2b-891d2432b2af None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.628789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3c3bd-ccc3-4a73-9d2b-891d2432b2af None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.663643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253f2cf-ddf7-432d-81d1-28f2386b9332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.664283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253f2cf-ddf7-432d-81d1-28f2386b9332 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.664559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253f2cf-ddf7-432d-81d1-28f2386b9332 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.664825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253f2cf-ddf7-432d-81d1-28f2386b9332 None admin] PATH_INFO: `/v3/projects/f0ce673894c2474fbaccb6becda86aa5/users/8b0ec6ad218d4286a5d0d0845262fad0/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.666840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253f2cf-ddf7-432d-81d1-28f2386b9332 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0ce673894c2474fbaccb6becda86aa5, user_id=8b0ec6ad218d4286a5d0d0845262fad0, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.668270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253f2cf-ddf7-432d-81d1-28f2386b9332 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.683805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 447/897] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:05 2022] PUT /identity/v3/projects/f0ce673894c2474fbaccb6becda86aa5/users/8b0ec6ad218d4286a5d0d0845262fad0/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:07.817095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a54d0-99ca-4023-b539-efc8a89978ba None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.818019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a54d0-99ca-4023-b539-efc8a89978ba None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.818019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a54d0-99ca-4023-b539-efc8a89978ba None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.821539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 451/898] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:17:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:07.828081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9462-fee5-41a7-a627-482d5a317ded None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.828546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9462-fee5-41a7-a627-482d5a317ded None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.828651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9462-fee5-41a7-a627-482d5a317ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.848862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-383e9462-fee5-41a7-a627-482d5a317ded None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:07.850065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-383e9462-fee5-41a7-a627-482d5a317ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:07.880297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:07.924740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 448/899] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:08.188475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a54d0-99ca-4023-b539-efc8a89978ba None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.301863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a54d0-99ca-4023-b539-efc8a89978ba None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.329031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.340230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aafa32-d5f5-4b23-be9d-11f8652a2a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.340857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aafa32-d5f5-4b23-be9d-11f8652a2a34 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.341140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aafa32-d5f5-4b23-be9d-11f8652a2a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.341413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aafa32-d5f5-4b23-be9d-11f8652a2a34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.351967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aafa32-d5f5-4b23-be9d-11f8652a2a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-63aafa32-d5f5-4b23-be9d-11f8652a2a34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:08.356748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:08.357822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:17:08.358667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 452/900] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:08.364066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e9462-fee5-41a7-a627-482d5a317ded None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.473519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e9462-fee5-41a7-a627-482d5a317ded None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.508675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.509275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.509514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.509805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.510484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:08.510781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.512158 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.520951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 449/901] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:08.529393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aafa32-d5f5-4b23-be9d-11f8652a2a34 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.639050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aafa32-d5f5-4b23-be9d-11f8652a2a34 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.675383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf900d3-742d-40a9-88c4-5a96be5aa473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.675997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf900d3-742d-40a9-88c4-5a96be5aa473 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.676277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf900d3-742d-40a9-88c4-5a96be5aa473 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.676550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf900d3-742d-40a9-88c4-5a96be5aa473 None admin] PATH_INFO: `/v3/users/8b0ec6ad218d4286a5d0d0845262fad0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.677802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf900d3-742d-40a9-88c4-5a96be5aa473 None admin] RBAC: Authorizing `identity:get_user(user_id=8b0ec6ad218d4286a5d0d0845262fad0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.679364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf900d3-742d-40a9-88c4-5a96be5aa473 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.680938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 453/902] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/users/8b0ec6ad218d4286a5d0d0845262fad0 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:08.690541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.722777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.796777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5de8b9-7d9b-496b-91af-0a32413ca3a1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.831510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0c435-789e-4e08-b1cf-37e447e23b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.832123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0c435-789e-4e08-b1cf-37e447e23b8f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.832356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0c435-789e-4e08-b1cf-37e447e23b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.832614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0c435-789e-4e08-b1cf-37e447e23b8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.908987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0c435-789e-4e08-b1cf-37e447e23b8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.910457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0c435-789e-4e08-b1cf-37e447e23b8f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.938933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.015863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 450/903] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:17:09.022339 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf900d3-742d-40a9-88c4-5a96be5aa473 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.087198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.134016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf900d3-742d-40a9-88c4-5a96be5aa473 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.168680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7d9ae-bda2-4b0f-9cb4-fa27e81f52d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.169282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7d9ae-bda2-4b0f-9cb4-fa27e81f52d3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.169553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7d9ae-bda2-4b0f-9cb4-fa27e81f52d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.169889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7d9ae-bda2-4b0f-9cb4-fa27e81f52d3 None admin] PATH_INFO: `/v3/projects/f0ce673894c2474fbaccb6becda86aa5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.171146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7d9ae-bda2-4b0f-9cb4-fa27e81f52d3 None admin] RBAC: Authorizing `identity:get_project(project_id=f0ce673894c2474fbaccb6becda86aa5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.172565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7d9ae-bda2-4b0f-9cb4-fa27e81f52d3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.174172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 454/904] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/projects/f0ce673894c2474fbaccb6becda86aa5 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.182901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0c435-789e-4e08-b1cf-37e447e23b8f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.287383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0c435-789e-4e08-b1cf-37e447e23b8f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.316000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.325353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4404fb-11f2-43fb-9837-12af468ac3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.325952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4404fb-11f2-43fb-9837-12af468ac3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.326266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4404fb-11f2-43fb-9837-12af468ac3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.326530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4404fb-11f2-43fb-9837-12af468ac3b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.327247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4404fb-11f2-43fb-9837-12af468ac3b9 None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=f0ce673894c2474fbaccb6becda86aa5, user.id=8b0ec6ad218d4286a5d0d0845262fad0) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.327554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4404fb-11f2-43fb-9837-12af468ac3b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.328966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4404fb-11f2-43fb-9837-12af468ac3b9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.335370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 451/905] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/role_assignments?user.id=8b0ec6ad218d4286a5d0d0845262fad0&scope.project.id=f0ce673894c2474fbaccb6becda86aa5&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.163644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=38 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.332347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:05.513547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbe97a-d36e-4137-beb0-a8cd5a537284 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.514274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbe97a-d36e-4137-beb0-a8cd5a537284 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.514274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbe97a-d36e-4137-beb0-a8cd5a537284 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.515821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 455/906] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:05.521608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527025d9-2bb5-4823-b2ac-eda241fe8f2f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:05.522245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527025d9-2bb5-4823-b2ac-eda241fe8f2f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:05.522245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527025d9-2bb5-4823-b2ac-eda241fe8f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:05.546565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-527025d9-2bb5-4823-b2ac-eda241fe8f2f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:05.547904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-527025d9-2bb5-4823-b2ac-eda241fe8f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:05.631682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 452/907] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:05.920652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbe97a-d36e-4137-beb0-a8cd5a537284 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.065036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbe97a-d36e-4137-beb0-a8cd5a537284 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.104295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cb32-029b-43cb-a543-004d932115de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.104885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cb32-029b-43cb-a543-004d932115de None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.105191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cb32-029b-43cb-a543-004d932115de None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.105503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cb32-029b-43cb-a543-004d932115de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.116801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323cb32-029b-43cb-a543-004d932115de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.118906 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323cb32-029b-43cb-a543-004d932115de None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c323cb32-029b-43cb-a543-004d932115de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:06.142636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:06.144036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:06.145084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:06.145084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:06.145084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:06.145084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 456/908] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:06.150752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527025d9-2bb5-4823-b2ac-eda241fe8f2f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.275918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527025d9-2bb5-4823-b2ac-eda241fe8f2f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.307282 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:06.313555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a445d-8d3c-475d-82f1-1a715cd28755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.314230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a445d-8d3c-475d-82f1-1a715cd28755 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.314586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a445d-8d3c-475d-82f1-1a715cd28755 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.314861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a445d-8d3c-475d-82f1-1a715cd28755 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.315509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a445d-8d3c-475d-82f1-1a715cd28755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:06.315811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a445d-8d3c-475d-82f1-1a715cd28755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.317215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a445d-8d3c-475d-82f1-1a715cd28755 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.331766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 453/909] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:06.343470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cb32-029b-43cb-a543-004d932115de None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:06.405001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:06.459762 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cb32-029b-43cb-a543-004d932115de None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:06.501560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd3410-386b-4a8f-85a2-227595ec9b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:06.502237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd3410-386b-4a8f-85a2-227595ec9b58 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:06.502561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd3410-386b-4a8f-85a2-227595ec9b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:06.502895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd3410-386b-4a8f-85a2-227595ec9b58 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:06.503518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd3410-386b-4a8f-85a2-227595ec9b58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:06.505052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd3410-386b-4a8f-85a2-227595ec9b58 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:06.538217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 457/910] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Nov 2 07:18:06 2022] POST /identity/v3/users => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:08.841444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347bba-4edb-4465-b797-a38a47cbfcdb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:08.842080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347bba-4edb-4465-b797-a38a47cbfcdb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:08.842080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347bba-4edb-4465-b797-a38a47cbfcdb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:08.843350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 454/911] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:08.849172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba569-61ea-49e8-bff8-d2332faafa87 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:08.849646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba569-61ea-49e8-bff8-d2332faafa87 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:08.849792 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ba569-61ea-49e8-bff8-d2332faafa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:08.871350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-172ba569-61ea-49e8-bff8-d2332faafa87 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:08.872711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-172ba569-61ea-49e8-bff8-d2332faafa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:08.968795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 458/912] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:09.274770 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347bba-4edb-4465-b797-a38a47cbfcdb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:09.392091 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347bba-4edb-4465-b797-a38a47cbfcdb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:09.431637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e164c2-40e9-4244-a8e0-89e7d58c501e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:09.432351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e164c2-40e9-4244-a8e0-89e7d58c501e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:09.432669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e164c2-40e9-4244-a8e0-89e7d58c501e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:09.432921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e164c2-40e9-4244-a8e0-89e7d58c501e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:09.446013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e164c2-40e9-4244-a8e0-89e7d58c501e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-23e164c2-40e9-4244-a8e0-89e7d58c501e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:09.451612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:09.452732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:09.454156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 455/913] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:18:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:09.459558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ba569-61ea-49e8-bff8-d2332faafa87 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:09.509070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:09.594181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ba569-61ea-49e8-bff8-d2332faafa87 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:09.630281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:09.630942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:09.631223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:09.631433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:09.632175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:09.632576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:09.634159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:09.642465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 459/914] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:18:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:09.648441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e164c2-40e9-4244-a8e0-89e7d58c501e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:09.710732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:09.776673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e164c2-40e9-4244-a8e0-89e7d58c501e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:09.817536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:09.818374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:09.818925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:09.819411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:09.833707 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:09.836744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:09.855236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:09.861802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:09.862924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:09.862924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:09.862924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:09.862924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 456/915] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:09.864305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:09.987181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:09.990775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea96cb0-dd78-4882-b7e2-ffed27ffefe1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.035182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.035827 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.036247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.036553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.037259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:10.037560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.039016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.052652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 460/916] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:10.058684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.172939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf627f6-e072-42c9-a26a-911884ad0297 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.175901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:10.212189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2840d-0209-4946-9274-662392928e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.213224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2840d-0209-4946-9274-662392928e71 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.213224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2840d-0209-4946-9274-662392928e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.213445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2840d-0209-4946-9274-662392928e71 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.222439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2840d-0209-4946-9274-662392928e71 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.224272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2840d-0209-4946-9274-662392928e71 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-97a2840d-0209-4946-9274-662392928e71 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:10.236787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:10.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:10.239937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:10.242251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 457/917] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:10.243963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.354120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba3b3-7413-4426-8754-7b38909fe8c0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.394879 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.395876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.396254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.396623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.397578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:10.398022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.399803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.423591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 461/918] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:10.424793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2840d-0209-4946-9274-662392928e71 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.538495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2840d-0209-4946-9274-662392928e71 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.580875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.581709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.582171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.582520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.598869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.600796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:10.619218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:10.620417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:10.621527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:10.621527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:10.621527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:10.622814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 458/919] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:10.630511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.652055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:10.738441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daec60f-823c-41ae-ad5b-6f70103597da None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.780770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.781389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.781971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.782263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.783059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:10.783525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.785313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.805353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 462/920] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:10.806850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:10.826511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:10.928194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050ac5-51d7-47e7-9db8-4359454010dc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:10.967089 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:10.967727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:10.968005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:10.968274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:10.976564 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:10.978010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:10.990241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:10.991367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:10.992333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 459/921] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:10.999800 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:11.098616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:11.123664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97490ad2-8daa-455e-a5de-37b7e70c26c3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:11.171587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6355f7-d47b-42bf-89a9-870e159f6c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:11.172301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6355f7-d47b-42bf-89a9-870e159f6c50 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:11.172640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6355f7-d47b-42bf-89a9-870e159f6c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:11.172961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6355f7-d47b-42bf-89a9-870e159f6c50 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:11.173743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6355f7-d47b-42bf-89a9-870e159f6c50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:11.174050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6355f7-d47b-42bf-89a9-870e159f6c50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:11.175547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6355f7-d47b-42bf-89a9-870e159f6c50 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:11.191566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 463/922] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:11.197850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:11.279590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:11.305102 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bac1f-804e-4def-83b0-b6109f3326a6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:11.339729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d12e7-4e45-4007-8c12-ef99575016ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:11.340419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d12e7-4e45-4007-8c12-ef99575016ec None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:11.340696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d12e7-4e45-4007-8c12-ef99575016ec None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:11.340974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d12e7-4e45-4007-8c12-ef99575016ec None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/db7ab631609f49249a010b1a8b03e002/roles/636139c60bd74dcaa328bf316f518d4e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:11.358215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d12e7-4e45-4007-8c12-ef99575016ec None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=db7ab631609f49249a010b1a8b03e002, role_id=636139c60bd74dcaa328bf316f518d4e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:11.359855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d12e7-4e45-4007-8c12-ef99575016ec None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:11.374827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 460/923] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:11 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/db7ab631609f49249a010b1a8b03e002/roles/636139c60bd74dcaa328bf316f518d4e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:18:13.611560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a81d3-fbf8-49b9-807e-85762dc75f9f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:13.612181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a81d3-fbf8-49b9-807e-85762dc75f9f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:13.612181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a81d3-fbf8-49b9-807e-85762dc75f9f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:13.613471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 464/924] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:13.621083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b45c-1e99-4ea9-9241-00660027e077 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:13.621531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b45c-1e99-4ea9-9241-00660027e077 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:13.621633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b45c-1e99-4ea9-9241-00660027e077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:13.645239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6a23b45c-1e99-4ea9-9241-00660027e077 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:13.646505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a23b45c-1e99-4ea9-9241-00660027e077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:13.723261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 461/925] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:13.996736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a81d3-fbf8-49b9-807e-85762dc75f9f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.099612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a81d3-fbf8-49b9-807e-85762dc75f9f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:14.135695 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b42ab-892d-483a-be5a-e29f19c65440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:14.136357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b42ab-892d-483a-be5a-e29f19c65440 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.136592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b42ab-892d-483a-be5a-e29f19c65440 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.136855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b42ab-892d-483a-be5a-e29f19c65440 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.144770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:14.148972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b42ab-892d-483a-be5a-e29f19c65440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e81b42ab-892d-483a-be5a-e29f19c65440 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:14.154716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:14.155835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:14.156671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 465/926] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Nov 2 07:18:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:14.162456 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23b45c-1e99-4ea9-9241-00660027e077 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.218666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:14.267618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23b45c-1e99-4ea9-9241-00660027e077 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:14.303136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:14.303737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.304012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.304491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.305164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:14.305417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:14.306813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:14.315203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 462/927] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Nov 2 07:18:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:14.321366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b42ab-892d-483a-be5a-e29f19c65440 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.429851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b42ab-892d-483a-be5a-e29f19c65440 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:14.468766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:14.469307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.469576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.469839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.479724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:14.481217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:14.497639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:14.498860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:14.499945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:14.499945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:14.499945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:14.504124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 466/928] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:14.505459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.621938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f65b-0af6-4975-8464-69b527092b95 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:14.639449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:14.658934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:14.659679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.660056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.660405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.661145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:14.661538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:14.663029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:14.675538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 463/929] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:14.681862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.736120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:14.790784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286ebac-9740-457e-a115-c24a7c7e312f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:14.827327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:14.828203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.828501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.828793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.837388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:14.839010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:14.852920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:14.854163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:14.855272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:14.859509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 467/930] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:14.860646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.972983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635d28-f0f8-44ee-93b7-d9f4e5e9fcab None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:15.010154 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:15.010799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:15.011178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:15.011478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:15.012313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:15.012677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:15.014530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:15.033649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 464/931] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Nov 2 07:18:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:15.041052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:15.154586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643eee0-836f-4695-bc67-08a7d74330f5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:15.166384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:15.193262 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cea6e-8691-400f-89fe-86a7656eab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:15.193885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cea6e-8691-400f-89fe-86a7656eab6b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:15.194195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cea6e-8691-400f-89fe-86a7656eab6b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:15.194491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cea6e-8691-400f-89fe-86a7656eab6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:15.277103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cea6e-8691-400f-89fe-86a7656eab6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:15.278555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cea6e-8691-400f-89fe-86a7656eab6b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:15.385088 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 468/932] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:15.391323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:15.428255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:15.496271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1501f-89fd-4cf8-b278-c3e783a10721 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:15.531216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:15.531825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:15.532152 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:15.532425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:15.543852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:15.546257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:15.562110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:15.563680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:15.564633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:15.564633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:15.564633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:15.564633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 465/933] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:15.569535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cea6e-8691-400f-89fe-86a7656eab6b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:15.583099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:15.683336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cea6e-8691-400f-89fe-86a7656eab6b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:15.722162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:15.722769 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:15.723048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:15.723320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:15.723967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:15.724273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:15.725657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:15.738186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 469/934] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:15.744994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:15.836486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:15.852550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4378c-f650-4f39-a155-211e2024d0ef None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:15.888211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:15.888810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:15.889128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:15.889362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:15.898048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:15.899711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:15.912174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:15.913340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:15.914482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 466/935] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:15.919675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:15.997060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:16.036164 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6d910-a6f1-4dc9-886d-c2d695e9bf14 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:16.073299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847c1dc-10ba-4cc5-95f5-822c63d2faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:16.074000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847c1dc-10ba-4cc5-95f5-822c63d2faba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:16.074332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847c1dc-10ba-4cc5-95f5-822c63d2faba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:16.074593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847c1dc-10ba-4cc5-95f5-822c63d2faba None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:16.075285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847c1dc-10ba-4cc5-95f5-822c63d2faba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:16.075611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847c1dc-10ba-4cc5-95f5-822c63d2faba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:16.077032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847c1dc-10ba-4cc5-95f5-822c63d2faba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:16.090188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 470/936] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:16.096598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:16.206596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d3310-fa03-4730-b321-dca206ea3009 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:16.243516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb607c-5409-4c84-995a-1a910e0f7167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:16.244194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb607c-5409-4c84-995a-1a910e0f7167 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:16.244494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb607c-5409-4c84-995a-1a910e0f7167 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:16.244786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb607c-5409-4c84-995a-1a910e0f7167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:16.245573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb607c-5409-4c84-995a-1a910e0f7167 None admin] RBAC: Adding query filter params (role.id=636139c60bd74dcaa328bf316f518d4e, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=db7ab631609f49249a010b1a8b03e002) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:16.246796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb607c-5409-4c84-995a-1a910e0f7167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:16.247384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb607c-5409-4c84-995a-1a910e0f7167 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:16.253869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 467/937] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:16 2022] GET /identity/v3/role_assignments?user.id=db7ab631609f49249a010b1a8b03e002&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=636139c60bd74dcaa328bf316f518d4e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:18.471301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1cba2-4b3c-46dc-9c80-288352e5779d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.471864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1cba2-4b3c-46dc-9c80-288352e5779d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.471864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1cba2-4b3c-46dc-9c80-288352e5779d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.473397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 471/938] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:18.479716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38946ea-af1b-4021-a7f7-aa931e7c65e0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.480202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38946ea-af1b-4021-a7f7-aa931e7c65e0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.480344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38946ea-af1b-4021-a7f7-aa931e7c65e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.500790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e38946ea-af1b-4021-a7f7-aa931e7c65e0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:18.502001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e38946ea-af1b-4021-a7f7-aa931e7c65e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:18.504927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:18.578394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 468/939] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:18.867595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1cba2-4b3c-46dc-9c80-288352e5779d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:18.977928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1cba2-4b3c-46dc-9c80-288352e5779d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.009436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.015949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378eb00-11e0-4b85-aabb-53012ec01618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.016786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378eb00-11e0-4b85-aabb-53012ec01618 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.017084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378eb00-11e0-4b85-aabb-53012ec01618 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.017362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378eb00-11e0-4b85-aabb-53012ec01618 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.027791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378eb00-11e0-4b85-aabb-53012ec01618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e378eb00-11e0-4b85-aabb-53012ec01618 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:19.032585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:19.033796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:19.034884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 472/940] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:19.040731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38946ea-af1b-4021-a7f7-aa931e7c65e0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.149638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38946ea-af1b-4021-a7f7-aa931e7c65e0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.186056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.186684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.187000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.187271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.187969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:19.188265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.189723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.198819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 469/941] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:19.204903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378eb00-11e0-4b85-aabb-53012ec01618 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.314411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378eb00-11e0-4b85-aabb-53012ec01618 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.357914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.358601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.358882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.359181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.371210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.372834 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:19.389110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:19.390256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:19.396313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:19.396313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:19.396313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:19.396313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 473/942] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:19.396697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.400946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.502438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea00c0-25b4-45a4-84fd-e7eefd5c4fec None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.539628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.540307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.540588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.540939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.541936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:19.542350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.544202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.558305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 470/943] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:19.564701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.617682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.678795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df739e-2c88-443c-9220-21afec26cbdb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.716173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.716870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.717148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.717427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.726849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.728754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:19.740870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:19.742042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:19.743215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 474/944] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:19.750206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.815931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.857077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dda90-6c53-455e-a96e-ffa594667c1f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.895209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.895824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.896102 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.896374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.897099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:19.897402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.898940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.917347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 471/945] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:19.925344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:20.039434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93063a20-2807-4403-b380-a4ed2a95eff9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:20.040902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:20.084990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:20.086336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:20.086605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:20.086873 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:20.097457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:20.098933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:20.114517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:20.120321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:20.121439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:20.121439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:20.121439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:20.121439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 475/946] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:20.121727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:20.240683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05162-a045-4c57-819b-9ae7ce1b49e4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:20.263528 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:20.277074 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:20.277678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:20.277955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:20.278263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:20.278902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:20.279198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:20.280553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:20.291568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 472/947] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:20.297358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:20.414107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051acec7-fdd5-4eed-9840-be2291e81f91 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:20.449453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:20.450152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:20.450507 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:20.450793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:20.457623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:20.459093 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:20.472403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:20.473534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:20.474856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 476/948] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:20.479682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:20.592344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf2e31-5b66-4507-ba0d-a336e92e9aa0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:20.630802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0247d53-c7d7-4063-96d9-3546cb6ebcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:20.631415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247d53-c7d7-4063-96d9-3546cb6ebcdc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:20.631689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247d53-c7d7-4063-96d9-3546cb6ebcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:20.631964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247d53-c7d7-4063-96d9-3546cb6ebcdc None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:20.632815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0247d53-c7d7-4063-96d9-3546cb6ebcdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:20.633269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0247d53-c7d7-4063-96d9-3546cb6ebcdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:20.634821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0247d53-c7d7-4063-96d9-3546cb6ebcdc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:20.647823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 473/949] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:20.653750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:20.671546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:20.766337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77134be-17df-4f94-bb74-545ae3c88b61 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:20.804386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500306e-c03c-4695-8d76-9abd96665e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:20.805206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500306e-c03c-4695-8d76-9abd96665e2e None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:20.805523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500306e-c03c-4695-8d76-9abd96665e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:20.805803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500306e-c03c-4695-8d76-9abd96665e2e None admin] PATH_INFO: `/v3/projects/c271e6d0d23647089c07648d4c38193d/users/db7ab631609f49249a010b1a8b03e002/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:20.808174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500306e-c03c-4695-8d76-9abd96665e2e None admin] RBAC: Authorizing `identity:create_grant(project_id=c271e6d0d23647089c07648d4c38193d, user_id=db7ab631609f49249a010b1a8b03e002, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:20.809862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500306e-c03c-4695-8d76-9abd96665e2e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:20.823351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 477/950] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:20 2022] PUT /identity/v3/projects/c271e6d0d23647089c07648d4c38193d/users/db7ab631609f49249a010b1a8b03e002/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:18:23.274035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb32688-5380-4fd5-9e07-92331a0f4939 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:23.274597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb32688-5380-4fd5-9e07-92331a0f4939 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:23.274597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb32688-5380-4fd5-9e07-92331a0f4939 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:23.276202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 474/951] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:23.281602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55cc7e-aba3-4b6d-9b9e-0a6160ea36bf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:23.282141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55cc7e-aba3-4b6d-9b9e-0a6160ea36bf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:23.282259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55cc7e-aba3-4b6d-9b9e-0a6160ea36bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:23.305330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0d55cc7e-aba3-4b6d-9b9e-0a6160ea36bf None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:23.306606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0d55cc7e-aba3-4b6d-9b9e-0a6160ea36bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:23.376585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 478/952] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:23.642686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb32688-5380-4fd5-9e07-92331a0f4939 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:23.693900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:23.755327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb32688-5380-4fd5-9e07-92331a0f4939 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:23.790607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c05c0-c1e4-4fde-b340-0cc6baead46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:23.791236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c05c0-c1e4-4fde-b340-0cc6baead46b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:23.791509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c05c0-c1e4-4fde-b340-0cc6baead46b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:23.791780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c05c0-c1e4-4fde-b340-0cc6baead46b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:23.803772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c05c0-c1e4-4fde-b340-0cc6baead46b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-948c05c0-c1e4-4fde-b340-0cc6baead46b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:23.809500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:23.810601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:23.811462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 475/953] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:23.817118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55cc7e-aba3-4b6d-9b9e-0a6160ea36bf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:23.820918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:23.934125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55cc7e-aba3-4b6d-9b9e-0a6160ea36bf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:23.973753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:23.974402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:23.974701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:23.975043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:23.976151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:23.976151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:23.977605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:23.986649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 479/954] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:23.995235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c05c0-c1e4-4fde-b340-0cc6baead46b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.111383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:24.114711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c05c0-c1e4-4fde-b340-0cc6baead46b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.152144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.152776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.153045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.153320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.164071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.165570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:24.179482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:24.180639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:24.181639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:24.181639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:24.181639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:24.185437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 476/955] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:24.186766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.267422 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:24.311692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736b09-d617-41dd-8c3d-b7f0d42e3786 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.353077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.353904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.354316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.354595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.355467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:24.355775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.357497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.370920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 480/956] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:24.376783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.483016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2a602-8f9e-41b7-93b4-7029a42db1cf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.520002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.520624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.520902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.521173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.529810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.531433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:24.544929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:24.546151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:24.547322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 477/957] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:24.552422 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.673521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce31780-1157-4779-93cd-60e825b8f63d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.701147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:24.711403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.712011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.712306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.712565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.713282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:24.713615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.715055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.735706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 481/958] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Nov 2 07:18:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:24.741788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.757820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:24.854240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246935d-2cb6-4204-b38c-4aa3a7566cf6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.894620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d258f-3100-4a1c-b4f4-b49f6e67ff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.898876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d258f-3100-4a1c-b4f4-b49f6e67ff4f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.898876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d258f-3100-4a1c-b4f4-b49f6e67ff4f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.898876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d258f-3100-4a1c-b4f4-b49f6e67ff4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.970660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d258f-3100-4a1c-b4f4-b49f6e67ff4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.972093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d258f-3100-4a1c-b4f4-b49f6e67ff4f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.985449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:25.076749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 478/959] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:25.083617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:25.204066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e985a-b26a-4c23-8f20-1cf4c6900edb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:25.242067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:25.242848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:25.243122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:25.243396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:25.253305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:25.254777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:25.269922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:25.270932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:25.271820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:25.271820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:25.271820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:25.271820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 482/960] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:25.277083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d258f-3100-4a1c-b4f4-b49f6e67ff4f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:25.384959 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d258f-3100-4a1c-b4f4-b49f6e67ff4f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:25.399173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:25.421354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:25.421963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:25.422279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:25.422571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:25.423355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:25.423637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:25.425287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:25.439151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 479/961] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:25.444925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:25.487930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:25.555712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1225798-edcb-475a-9a82-7dde8658ab64 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:25.594495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:25.595144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:25.595402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:25.595696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:25.603318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:25.604787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:25.618191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:25.619388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 07:18:25.625480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 483/962] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:25.625972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:25.734196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddd246-f418-4b8b-aa5b-ee8af34eeb1d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:25.769654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f721c92-a9cc-41d5-af60-94e52dbe794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:25.770318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f721c92-a9cc-41d5-af60-94e52dbe794b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:25.770617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f721c92-a9cc-41d5-af60-94e52dbe794b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:25.770913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f721c92-a9cc-41d5-af60-94e52dbe794b None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:25.771658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f721c92-a9cc-41d5-af60-94e52dbe794b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:25.771978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f721c92-a9cc-41d5-af60-94e52dbe794b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:25.773663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f721c92-a9cc-41d5-af60-94e52dbe794b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:25.786737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 480/963] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:25.792760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:25.900686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:25.907280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decdd99-c6fd-4151-8750-f9c8ba2bb851 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:25.944834 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1b4eb-bbab-4e7d-aab5-ef2c17515e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:25.945590 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1b4eb-bbab-4e7d-aab5-ef2c17515e91 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:25.945867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1b4eb-bbab-4e7d-aab5-ef2c17515e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:25.946221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1b4eb-bbab-4e7d-aab5-ef2c17515e91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:25.946952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1b4eb-bbab-4e7d-aab5-ef2c17515e91 None admin] RBAC: Adding query filter params (role.id=f5e3c85c776c4e7392fac5e1418c0bbc, scope.project.id=c271e6d0d23647089c07648d4c38193d, user.id=db7ab631609f49249a010b1a8b03e002) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:25.947318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1b4eb-bbab-4e7d-aab5-ef2c17515e91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:25.948786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1b4eb-bbab-4e7d-aab5-ef2c17515e91 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:25.954992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 484/964] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:25 2022] GET /identity/v3/role_assignments?user.id=db7ab631609f49249a010b1a8b03e002&scope.project.id=c271e6d0d23647089c07648d4c38193d&role.id=f5e3c85c776c4e7392fac5e1418c0bbc => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:28.116751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56149eaa-47ca-4e45-a9c8-0274ea7d8871 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.117788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56149eaa-47ca-4e45-a9c8-0274ea7d8871 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.117788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56149eaa-47ca-4e45-a9c8-0274ea7d8871 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.119739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 481/965] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:28.128399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca40f1-aef1-496d-8cd2-0f403df06688 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.128815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca40f1-aef1-496d-8cd2-0f403df06688 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.128998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca40f1-aef1-496d-8cd2-0f403df06688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.154882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-37ca40f1-aef1-496d-8cd2-0f403df06688 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:28.156075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-37ca40f1-aef1-496d-8cd2-0f403df06688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:28.245313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 485/966] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:28.526849 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56149eaa-47ca-4e45-a9c8-0274ea7d8871 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.562885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.644918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56149eaa-47ca-4e45-a9c8-0274ea7d8871 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.682192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.682823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.683108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.683355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.683964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.685347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:28.694581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:28.695784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:28.696691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 482/967] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:28.702961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca40f1-aef1-496d-8cd2-0f403df06688 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.799316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.810116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca40f1-aef1-496d-8cd2-0f403df06688 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.848348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb11ce-5b2c-42cb-845a-31dcdbbda13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.848938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb11ce-5b2c-42cb-845a-31dcdbbda13a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.849209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb11ce-5b2c-42cb-845a-31dcdbbda13a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.849475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb11ce-5b2c-42cb-845a-31dcdbbda13a None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.850132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb11ce-5b2c-42cb-845a-31dcdbbda13a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:28.850420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb11ce-5b2c-42cb-845a-31dcdbbda13a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.851820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb11ce-5b2c-42cb-845a-31dcdbbda13a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.856630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 486/968] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:28.863175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.964384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.971013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a0017-6fab-4faf-83b6-6d3d4891ecc2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:29.005512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d305e4c-bb6c-40ce-a7c3-e55f50346f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:29.006220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d305e4c-bb6c-40ce-a7c3-e55f50346f8e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:29.006461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d305e4c-bb6c-40ce-a7c3-e55f50346f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:29.006660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d305e4c-bb6c-40ce-a7c3-e55f50346f8e None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:29.007351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d305e4c-bb6c-40ce-a7c3-e55f50346f8e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:29.007612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d305e4c-bb6c-40ce-a7c3-e55f50346f8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:29.008967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d305e4c-bb6c-40ce-a7c3-e55f50346f8e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:29.013993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 483/969] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:31.144163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc90c7-27e2-499d-9d4e-73ce371e29be None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.144724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc90c7-27e2-499d-9d4e-73ce371e29be None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.144724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc90c7-27e2-499d-9d4e-73ce371e29be None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.146063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 487/970] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:31.152677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132561f-223f-412a-a8e4-fb332c0c464d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.153383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132561f-223f-412a-a8e4-fb332c0c464d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.153559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132561f-223f-412a-a8e4-fb332c0c464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.178387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b132561f-223f-412a-a8e4-fb332c0c464d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:31.180000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b132561f-223f-412a-a8e4-fb332c0c464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:31.251317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 484/971] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:31.528472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc90c7-27e2-499d-9d4e-73ce371e29be None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:31.649159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc90c7-27e2-499d-9d4e-73ce371e29be None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:31.685442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad671d-e455-4f40-b27f-fdc14b177750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:31.686034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad671d-e455-4f40-b27f-fdc14b177750 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.686344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad671d-e455-4f40-b27f-fdc14b177750 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.686617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad671d-e455-4f40-b27f-fdc14b177750 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.687278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad671d-e455-4f40-b27f-fdc14b177750 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:31.688650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad671d-e455-4f40-b27f-fdc14b177750 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:31.705639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 488/972] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Nov 2 07:18:31 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:33.828640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8bd0-9b8f-48f5-bc31-9351408e43ab None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:33.829519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8bd0-9b8f-48f5-bc31-9351408e43ab None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:33.829519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8bd0-9b8f-48f5-bc31-9351408e43ab None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:33.831761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 485/973] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:33.837839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33d0ab-516b-4f95-9d61-94b6a5c0ed48 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:33.838310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33d0ab-516b-4f95-9d61-94b6a5c0ed48 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:33.838399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33d0ab-516b-4f95-9d61-94b6a5c0ed48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:33.857835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7e33d0ab-516b-4f95-9d61-94b6a5c0ed48 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:33.859032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e33d0ab-516b-4f95-9d61-94b6a5c0ed48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:33.868182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:33.945400 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 489/974] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:18:34.208807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8bd0-9b8f-48f5-bc31-9351408e43ab None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:34.328442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:34.341272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8bd0-9b8f-48f5-bc31-9351408e43ab None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:34.376952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:34.377577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.377846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.378140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.378722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:34.380094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:34.389880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:34.391005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:34.391854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 486/975] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Nov 2 07:18:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:34.397708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33d0ab-516b-4f95-9d61-94b6a5c0ed48 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:34.499211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33d0ab-516b-4f95-9d61-94b6a5c0ed48 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:34.533403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:34.533988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.534292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.534562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.535180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:34.535476 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:34.536867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:34.542066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 490/976] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Nov 2 07:18:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:34.548497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:34.665665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce785300-587a-4e61-99c0-5dc9ddf603f4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:34.702540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c281049-9b78-40e6-9dcb-db0cc5333aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:34.703141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c281049-9b78-40e6-9dcb-db0cc5333aef None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.703420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c281049-9b78-40e6-9dcb-db0cc5333aef None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.703690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c281049-9b78-40e6-9dcb-db0cc5333aef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.704373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c281049-9b78-40e6-9dcb-db0cc5333aef None admin] RBAC: Adding query filter params (interface=public, service_id=37519d34b28f4f2a8239dfc5d9a88afa, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:34.704676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c281049-9b78-40e6-9dcb-db0cc5333aef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:34.706052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c281049-9b78-40e6-9dcb-db0cc5333aef None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:34.711650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 487/977] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Nov 2 07:18:34 2022] GET /identity/v3/endpoints?service_id=37519d34b28f4f2a8239dfc5d9a88afa&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:34.718440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:34.724505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:34.830878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a087a26-6937-4cde-8aa3-4f860f166d91 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:34.874494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ea896-3cc8-49d5-bff5-fad2a5554c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:34.875118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ea896-3cc8-49d5-bff5-fad2a5554c56 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.875354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ea896-3cc8-49d5-bff5-fad2a5554c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.875624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ea896-3cc8-49d5-bff5-fad2a5554c56 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.876229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ea896-3cc8-49d5-bff5-fad2a5554c56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:34.877610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ea896-3cc8-49d5-bff5-fad2a5554c56 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:34.884689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 491/978] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Nov 2 07:18:34 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:18:37.002384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524dcc-4195-4bb7-83a5-c4ee0dbe9fed None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.003313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524dcc-4195-4bb7-83a5-c4ee0dbe9fed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.003313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524dcc-4195-4bb7-83a5-c4ee0dbe9fed None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.005419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 488/979] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:18:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:37.011111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600dbf-b159-4e96-9af3-22d551478644 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.011561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600dbf-b159-4e96-9af3-22d551478644 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.011667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600dbf-b159-4e96-9af3-22d551478644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.031419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f7600dbf-b159-4e96-9af3-22d551478644 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:37.032616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f7600dbf-b159-4e96-9af3-22d551478644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:37.098983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:37.105603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 492/980] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:37.378930 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97524dcc-4195-4bb7-83a5-c4ee0dbe9fed None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:37.405850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:37.493742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97524dcc-4195-4bb7-83a5-c4ee0dbe9fed None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:37.528156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:37.528763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.529040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.529265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.529864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:37.531254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:37.540348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:37.541429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Nov 02 07:18:37.542605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 489/981] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Nov 2 07:18:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:37.547632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7600dbf-b159-4e96-9af3-22d551478644 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:37.664706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7600dbf-b159-4e96-9af3-22d551478644 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:37.700939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac11998-c45b-416a-9036-57a11de8d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:37.701532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac11998-c45b-416a-9036-57a11de8d988 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.701807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac11998-c45b-416a-9036-57a11de8d988 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.702078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac11998-c45b-416a-9036-57a11de8d988 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.702728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac11998-c45b-416a-9036-57a11de8d988 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:37.703041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac11998-c45b-416a-9036-57a11de8d988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:37.704395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac11998-c45b-416a-9036-57a11de8d988 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:37.709540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 493/982] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Nov 2 07:18:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:37.716840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:37.807353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:37.830478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0840b44-ecdb-41db-a1f6-f0b3edbe95f4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:37.866217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958371-12c7-4620-a7e9-c96983598b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:37.866974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958371-12c7-4620-a7e9-c96983598b07 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.867470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958371-12c7-4620-a7e9-c96983598b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.867746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958371-12c7-4620-a7e9-c96983598b07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.868414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958371-12c7-4620-a7e9-c96983598b07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:37.869823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958371-12c7-4620-a7e9-c96983598b07 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:37.895678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 490/983] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:18:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:47.706532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=69 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:47.888700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:42.204950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb545-1803-4a38-aaaf-e136fc0fda2d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:42.204950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb545-1803-4a38-aaaf-e136fc0fda2d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:42.205448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb545-1803-4a38-aaaf-e136fc0fda2d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:42.212898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76635-e93c-404e-adf9-861f989900a9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:42.213315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76635-e93c-404e-adf9-861f989900a9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:42.213405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76635-e93c-404e-adf9-861f989900a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:42.216159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 494/984] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:19:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:42.331772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1cc76635-e93c-404e-adf9-861f989900a9 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:42.333482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1cc76635-e93c-404e-adf9-861f989900a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:42.526248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 491/985] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:42.818624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e70c3-d97b-4f96-9219-dfcc0dae5b43 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:42.819129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e70c3-d97b-4f96-9219-dfcc0dae5b43 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:42.819532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e70c3-d97b-4f96-9219-dfcc0dae5b43 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:42.821040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 495/986] 10.222.0.25 () {54 vars in 965 bytes} [Wed Nov 2 07:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:42.827464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a6db1-ee03-44ed-ad00-60ef1ff22659 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:42.827867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a6db1-ee03-44ed-ad00-60ef1ff22659 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:42.828242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a6db1-ee03-44ed-ad00-60ef1ff22659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:42.912945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f50a6db1-ee03-44ed-ad00-60ef1ff22659 None None] MFA Rules not processed for user `c4ad0e5567bd452eadd14ed6ab1edc21`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:42.912945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f50a6db1-ee03-44ed-ad00-60ef1ff22659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:42.934845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:43.001799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 492/987] 10.222.0.25 () {58 vars in 1062 bytes} [Wed Nov 2 07:19:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:43.011439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e70c3-d97b-4f96-9219-dfcc0dae5b43 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:43.013310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5a4e70c3-d97b-4f96-9219-dfcc0dae5b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:43.138398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e70c3-d97b-4f96-9219-dfcc0dae5b43 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:43.219636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001277a-29b5-432c-bf92-1e025f67267b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:43.220323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001277a-29b5-432c-bf92-1e025f67267b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:43.220496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001277a-29b5-432c-bf92-1e025f67267b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:43.220783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001277a-29b5-432c-bf92-1e025f67267b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:43.222423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8001277a-29b5-432c-bf92-1e025f67267b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:43.281069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:43.339662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001277a-29b5-432c-bf92-1e025f67267b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:43.341208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001277a-29b5-432c-bf92-1e025f67267b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:43.360996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 496/988] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:45.811323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fc777-d56b-45cf-801b-0e21decb06c5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:45.811756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fc777-d56b-45cf-801b-0e21decb06c5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:45.812050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fc777-d56b-45cf-801b-0e21decb06c5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:45.813665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 493/989] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:19:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:45.819430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36461c6e-fda4-43c6-a147-883e38f64e23 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:45.819860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36461c6e-fda4-43c6-a147-883e38f64e23 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:45.820096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36461c6e-fda4-43c6-a147-883e38f64e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:45.891950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-36461c6e-fda4-43c6-a147-883e38f64e23 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:45.893210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36461c6e-fda4-43c6-a147-883e38f64e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:45.996379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 497/990] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:46.272731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fc777-d56b-45cf-801b-0e21decb06c5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:46.274318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d51fc777-d56b-45cf-801b-0e21decb06c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:46.382948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fc777-d56b-45cf-801b-0e21decb06c5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:46.456132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:46.458804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee9b1-85de-41b6-8037-264121651458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:46.459482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee9b1-85de-41b6-8037-264121651458 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:46.459799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee9b1-85de-41b6-8037-264121651458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:46.460072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee9b1-85de-41b6-8037-264121651458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:46.462546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ce5ee9b1-85de-41b6-8037-264121651458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:46.579531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee9b1-85de-41b6-8037-264121651458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:46.581013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee9b1-85de-41b6-8037-264121651458 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:46.610856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 494/991] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:49.069145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35526bf3-8b4a-4361-ab89-7b57af8f9720 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:49.069708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35526bf3-8b4a-4361-ab89-7b57af8f9720 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:49.069708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35526bf3-8b4a-4361-ab89-7b57af8f9720 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:49.071073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 498/992] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:49.077433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830b0b3-ace4-4ba0-b20b-b8148406bdbf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:49.078235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830b0b3-ace4-4ba0-b20b-b8148406bdbf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:49.078392 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830b0b3-ace4-4ba0-b20b-b8148406bdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:49.157336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a830b0b3-ace4-4ba0-b20b-b8148406bdbf None None] MFA Rules not processed for user `97212451ad45468b86698f04b5189e88`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:49.158591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a830b0b3-ace4-4ba0-b20b-b8148406bdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:49.212840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:49.266921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 495/993] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:49.553802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35526bf3-8b4a-4361-ab89-7b57af8f9720 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:49.556281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35526bf3-8b4a-4361-ab89-7b57af8f9720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:49.570887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:49.670719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35526bf3-8b4a-4361-ab89-7b57af8f9720 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:49.757640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838104b2-f489-44b9-9c92-7f31fc569c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:49.758302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838104b2-f489-44b9-9c92-7f31fc569c06 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:49.758582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838104b2-f489-44b9-9c92-7f31fc569c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:49.758855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838104b2-f489-44b9-9c92-7f31fc569c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:49.760468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-838104b2-f489-44b9-9c92-7f31fc569c06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:49.828078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:49.880681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838104b2-f489-44b9-9c92-7f31fc569c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:49.882379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838104b2-f489-44b9-9c92-7f31fc569c06 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:49.905438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 499/994] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:52.579353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794237b6-a75a-4905-95ca-375e598283e9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.580369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794237b6-a75a-4905-95ca-375e598283e9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.580369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794237b6-a75a-4905-95ca-375e598283e9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.582553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 496/995] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:19:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:52.588431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48adada-97cc-414c-98a9-ffcd8b4d90ab None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.588865 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48adada-97cc-414c-98a9-ffcd8b4d90ab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.588961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48adada-97cc-414c-98a9-ffcd8b4d90ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.661348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a48adada-97cc-414c-98a9-ffcd8b4d90ab None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:52.662650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a48adada-97cc-414c-98a9-ffcd8b4d90ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:52.765120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 500/996] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:53.047789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794237b6-a75a-4905-95ca-375e598283e9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:53.050423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-794237b6-a75a-4905-95ca-375e598283e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:53.162242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794237b6-a75a-4905-95ca-375e598283e9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:53.242578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec470e-16a3-4cbd-9b11-a42305c5315d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:53.243098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec470e-16a3-4cbd-9b11-a42305c5315d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:53.243373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec470e-16a3-4cbd-9b11-a42305c5315d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:53.243643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec470e-16a3-4cbd-9b11-a42305c5315d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:53.245228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-55ec470e-16a3-4cbd-9b11-a42305c5315d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:53.252134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:53.371006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec470e-16a3-4cbd-9b11-a42305c5315d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:53.372422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec470e-16a3-4cbd-9b11-a42305c5315d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:53.391693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 497/997] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:56.196150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef867621-c945-45d1-9653-528b2400fd78 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:56.196774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef867621-c945-45d1-9653-528b2400fd78 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:56.196774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef867621-c945-45d1-9653-528b2400fd78 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:56.198031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 501/998] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:56.203606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf40ae9-0a36-44b5-acba-36e8588b3f38 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:56.204025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf40ae9-0a36-44b5-acba-36e8588b3f38 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:56.204109 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf40ae9-0a36-44b5-acba-36e8588b3f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:56.273663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fbf40ae9-0a36-44b5-acba-36e8588b3f38 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:56.275280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fbf40ae9-0a36-44b5-acba-36e8588b3f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:56.343717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:56.389037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 498/999] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:56.664927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef867621-c945-45d1-9653-528b2400fd78 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:56.666600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ef867621-c945-45d1-9653-528b2400fd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:56.688351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:56.778344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef867621-c945-45d1-9653-528b2400fd78 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:56.853506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67881b77-f84f-4c1f-876b-63e253fba297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:56.854153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881b77-f84f-4c1f-876b-63e253fba297 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:56.854434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881b77-f84f-4c1f-876b-63e253fba297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:56.854706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881b77-f84f-4c1f-876b-63e253fba297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:56.856350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-67881b77-f84f-4c1f-876b-63e253fba297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:56.930620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:56.969543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67881b77-f84f-4c1f-876b-63e253fba297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:56.970995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67881b77-f84f-4c1f-876b-63e253fba297 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:56.989210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 502/1000] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:59.185310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe7751-4055-40a8-ac2c-6a4d8090d6c3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:59.185844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe7751-4055-40a8-ac2c-6a4d8090d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:59.185844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe7751-4055-40a8-ac2c-6a4d8090d6c3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:59.187114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 499/1001] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:59.192548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106f822-32ff-4e86-803e-f93178eb7ce2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:59.192992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106f822-32ff-4e86-803e-f93178eb7ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:59.193081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106f822-32ff-4e86-803e-f93178eb7ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:59.260682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3106f822-32ff-4e86-803e-f93178eb7ce2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:59.261880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3106f822-32ff-4e86-803e-f93178eb7ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:59.372633 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 503/1002] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:59.650473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe7751-4055-40a8-ac2c-6a4d8090d6c3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:59.652028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9cbe7751-4055-40a8-ac2c-6a4d8090d6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:59.760699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe7751-4055-40a8-ac2c-6a4d8090d6c3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:59.841566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692d2ce-cc60-4a02-accd-9255507220bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:59.842213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692d2ce-cc60-4a02-accd-9255507220bb service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:59.842492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692d2ce-cc60-4a02-accd-9255507220bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:59.842742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692d2ce-cc60-4a02-accd-9255507220bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:59.844363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5692d2ce-cc60-4a02-accd-9255507220bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:59.862358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:59.962827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692d2ce-cc60-4a02-accd-9255507220bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:59.964264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692d2ce-cc60-4a02-accd-9255507220bb service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:59.983227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 500/1003] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:02.241185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a1e7-c752-4b1a-9789-8fcb7b2ddced None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.241710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a1e7-c752-4b1a-9789-8fcb7b2ddced None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.241817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a1e7-c752-4b1a-9789-8fcb7b2ddced None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.243205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 504/1004] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:02.248745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4f544-a7e6-4c83-a180-290695fabba3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.249193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4f544-a7e6-4c83-a180-290695fabba3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.249281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4f544-a7e6-4c83-a180-290695fabba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.318508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-54d4f544-a7e6-4c83-a180-290695fabba3 None None] MFA Rules not processed for user `97212451ad45468b86698f04b5189e88`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:02.319786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-54d4f544-a7e6-4c83-a180-290695fabba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:02.390479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:02.423935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 501/1005] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:20:02.719259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c247f-429c-4080-add3-daefeafab3b2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.719259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c247f-429c-4080-add3-daefeafab3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.719259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c247f-429c-4080-add3-daefeafab3b2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.720509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 505/1006] 10.222.0.25 () {54 vars in 965 bytes} [Wed Nov 2 07:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:02.726501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a662f-323b-4495-9691-293dc0890f3e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.726778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a662f-323b-4495-9691-293dc0890f3e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.727033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a662f-323b-4495-9691-293dc0890f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.810979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-709a662f-323b-4495-9691-293dc0890f3e None None] MFA Rules not processed for user `c4ad0e5567bd452eadd14ed6ab1edc21`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:02.812049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-709a662f-323b-4495-9691-293dc0890f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:02.890937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 502/1007] 10.222.0.25 () {58 vars in 1062 bytes} [Wed Nov 2 07:20:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:02.900619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c247f-429c-4080-add3-daefeafab3b2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:02.902231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9f6c247f-429c-4080-add3-daefeafab3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:02.936927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:03.013927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c247f-429c-4080-add3-daefeafab3b2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:03.090733 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b705e1-79e3-48ac-aec8-d54f531035da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:03.091261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b705e1-79e3-48ac-aec8-d54f531035da service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.091536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b705e1-79e3-48ac-aec8-d54f531035da service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.091810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b705e1-79e3-48ac-aec8-d54f531035da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.093406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-55b705e1-79e3-48ac-aec8-d54f531035da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:03.171566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:03.205414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b705e1-79e3-48ac-aec8-d54f531035da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:03.206908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b705e1-79e3-48ac-aec8-d54f531035da service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:03.232435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 506/1008] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:06.036394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07e225-d742-4701-a727-03b0b1032d78 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:06.037580 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07e225-d742-4701-a727-03b0b1032d78 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:06.037580 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07e225-d742-4701-a727-03b0b1032d78 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:06.038766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 503/1009] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:06.044269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cefcef-5b6a-4896-8211-206b29fc02b9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:06.044724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cefcef-5b6a-4896-8211-206b29fc02b9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:06.044813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cefcef-5b6a-4896-8211-206b29fc02b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:06.111810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a1cefcef-5b6a-4896-8211-206b29fc02b9 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:06.112933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1cefcef-5b6a-4896-8211-206b29fc02b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:06.210932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 507/1010] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:06.491157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07e225-d742-4701-a727-03b0b1032d78 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:06.493601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7e07e225-d742-4701-a727-03b0b1032d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:06.571253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:06.616193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07e225-d742-4701-a727-03b0b1032d78 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:06.714952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640eb21-5191-4c26-9a6c-1b3fc854fa15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:06.715559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640eb21-5191-4c26-9a6c-1b3fc854fa15 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:06.715871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640eb21-5191-4c26-9a6c-1b3fc854fa15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:06.716139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640eb21-5191-4c26-9a6c-1b3fc854fa15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:06.717783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0640eb21-5191-4c26-9a6c-1b3fc854fa15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:06.850687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640eb21-5191-4c26-9a6c-1b3fc854fa15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:06.852057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640eb21-5191-4c26-9a6c-1b3fc854fa15 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:06.862363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:06.871662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 504/1011] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:09.063943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d9a4-0dce-47eb-808f-77513629d025 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:09.064554 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d9a4-0dce-47eb-808f-77513629d025 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:09.064554 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6d9a4-0dce-47eb-808f-77513629d025 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:09.065833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 508/1012] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:09.072313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2927eb-493e-41fd-a04c-c067fe1876d5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:09.072777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2927eb-493e-41fd-a04c-c067fe1876d5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:09.072884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2927eb-493e-41fd-a04c-c067fe1876d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:09.146957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8d2927eb-493e-41fd-a04c-c067fe1876d5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:09.147938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8d2927eb-493e-41fd-a04c-c067fe1876d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:09.257337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 505/1013] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:09.543245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6d9a4-0dce-47eb-808f-77513629d025 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:09.545642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b8c6d9a4-0dce-47eb-808f-77513629d025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:09.597743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:09.665453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6d9a4-0dce-47eb-808f-77513629d025 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:09.742316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062061f3-1885-4b06-b31e-88fa5eb74fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:09.743212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062061f3-1885-4b06-b31e-88fa5eb74fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:09.743481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062061f3-1885-4b06-b31e-88fa5eb74fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:09.743742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062061f3-1885-4b06-b31e-88fa5eb74fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:09.745328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-062061f3-1885-4b06-b31e-88fa5eb74fc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:09.839143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:09.866400 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062061f3-1885-4b06-b31e-88fa5eb74fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:09.867854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062061f3-1885-4b06-b31e-88fa5eb74fc4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:09.887203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 509/1014] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:12.340149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab906109-e5ef-4974-8b0d-8267cc628160 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:12.340804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab906109-e5ef-4974-8b0d-8267cc628160 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:12.340804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab906109-e5ef-4974-8b0d-8267cc628160 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:12.342907 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 506/1015] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:12.349337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d338f-9610-4481-bfaa-6963bf027ea4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:12.350281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d338f-9610-4481-bfaa-6963bf027ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:12.350804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d338f-9610-4481-bfaa-6963bf027ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:12.434341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-709d338f-9610-4481-bfaa-6963bf027ea4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:12.435552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-709d338f-9610-4481-bfaa-6963bf027ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:12.541754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 510/1016] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:12.821393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab906109-e5ef-4974-8b0d-8267cc628160 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:12.823829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab906109-e5ef-4974-8b0d-8267cc628160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:12.911153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:12.941337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab906109-e5ef-4974-8b0d-8267cc628160 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:13.040281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ca835-4b63-46fc-a86a-e64b107ae18b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:13.040890 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ca835-4b63-46fc-a86a-e64b107ae18b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:13.041189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ca835-4b63-46fc-a86a-e64b107ae18b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:13.041453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ca835-4b63-46fc-a86a-e64b107ae18b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:13.043066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fe3ca835-4b63-46fc-a86a-e64b107ae18b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:13.153699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ca835-4b63-46fc-a86a-e64b107ae18b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:13.155177 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ca835-4b63-46fc-a86a-e64b107ae18b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:13.173194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 507/1017] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:17.465735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d1f4-3088-49ba-be20-29344d91dcf6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:17.466328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d1f4-3088-49ba-be20-29344d91dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:17.466328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d1f4-3088-49ba-be20-29344d91dcf6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:17.467743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 511/1018] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:17.473304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ca5e3-0c90-445a-b570-9689ff53b0bf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:17.473790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ca5e3-0c90-445a-b570-9689ff53b0bf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:17.473790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ca5e3-0c90-445a-b570-9689ff53b0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:17.479543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:17.543165 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0f0ca5e3-0c90-445a-b570-9689ff53b0bf None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:17.544357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0f0ca5e3-0c90-445a-b570-9689ff53b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:17.646037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 508/1019] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:17.923392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070d1f4-3088-49ba-be20-29344d91dcf6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:17.924973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e070d1f4-3088-49ba-be20-29344d91dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:17.977510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:18.035905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070d1f4-3088-49ba-be20-29344d91dcf6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:18.111915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4328ac9-4b76-4c05-98b1-eea9e8c2a95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:18.112530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328ac9-4b76-4c05-98b1-eea9e8c2a95a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:18.112819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328ac9-4b76-4c05-98b1-eea9e8c2a95a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:18.113092 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328ac9-4b76-4c05-98b1-eea9e8c2a95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:18.114772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b4328ac9-4b76-4c05-98b1-eea9e8c2a95a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:18.213709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:18.231026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4328ac9-4b76-4c05-98b1-eea9e8c2a95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:18.232479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4328ac9-4b76-4c05-98b1-eea9e8c2a95a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:18.250640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 512/1020] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:21.061666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfc90-7e5a-453f-80cf-68c562129e68 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:21.062252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfc90-7e5a-453f-80cf-68c562129e68 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:21.062252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfc90-7e5a-453f-80cf-68c562129e68 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:21.063603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 509/1021] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:21.069091 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f5aa-feaa-4174-8aee-61b4220e1927 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:21.069993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f5aa-feaa-4174-8aee-61b4220e1927 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:21.069993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f5aa-feaa-4174-8aee-61b4220e1927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:21.137457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fab6f5aa-feaa-4174-8aee-61b4220e1927 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:21.138688 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fab6f5aa-feaa-4174-8aee-61b4220e1927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:21.242008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 513/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:21.528066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cfc90-7e5a-453f-80cf-68c562129e68 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:21.529670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6e7cfc90-7e5a-453f-80cf-68c562129e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:21.633657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:21.653060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cfc90-7e5a-453f-80cf-68c562129e68 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:21.733860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d5690-ebd0-4575-9b48-59fba999c81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:21.734509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d5690-ebd0-4575-9b48-59fba999c81f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:21.734787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d5690-ebd0-4575-9b48-59fba999c81f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:21.735058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d5690-ebd0-4575-9b48-59fba999c81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:21.736650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ed7d5690-ebd0-4575-9b48-59fba999c81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:21.853945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d5690-ebd0-4575-9b48-59fba999c81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:21.855422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d5690-ebd0-4575-9b48-59fba999c81f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:21.874818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 510/1023] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:26.051319 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd136886-b579-4ffd-bf11-a3a3b203896e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:26.051929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd136886-b579-4ffd-bf11-a3a3b203896e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:26.051929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd136886-b579-4ffd-bf11-a3a3b203896e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:26.053130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 514/1024] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:26.059047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d703a7-e5db-447e-be86-1d5935ce3984 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:26.059555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d703a7-e5db-447e-be86-1d5935ce3984 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:26.059555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d703a7-e5db-447e-be86-1d5935ce3984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:26.075565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:26.136278 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-54d703a7-e5db-447e-be86-1d5935ce3984 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:26.136605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-54d703a7-e5db-447e-be86-1d5935ce3984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:26.243098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 511/1025] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:26.529029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd136886-b579-4ffd-bf11-a3a3b203896e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:26.530623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd136886-b579-4ffd-bf11-a3a3b203896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:26.595801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:26.647219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd136886-b579-4ffd-bf11-a3a3b203896e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:26.733066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e847fc4-b739-445a-9309-990812e26261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:26.733666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847fc4-b739-445a-9309-990812e26261 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:26.733935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847fc4-b739-445a-9309-990812e26261 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:26.734230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847fc4-b739-445a-9309-990812e26261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:26.735817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e847fc4-b739-445a-9309-990812e26261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:26.847226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:26.852436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e847fc4-b739-445a-9309-990812e26261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:26.854292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e847fc4-b739-445a-9309-990812e26261 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:26.874034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 515/1026] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:20:31.833612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079f901-aaac-4adc-bd61-db70401bbb20 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:31.834104 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079f901-aaac-4adc-bd61-db70401bbb20 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:31.834202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079f901-aaac-4adc-bd61-db70401bbb20 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:31.836045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 512/1027] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:31.847104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3943ec-4f11-40f8-9aa9-f51bc3663f01 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:31.847902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3943ec-4f11-40f8-9aa9-f51bc3663f01 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:31.848345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3943ec-4f11-40f8-9aa9-f51bc3663f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:31.992069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1c3943ec-4f11-40f8-9aa9-f51bc3663f01 None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:31.993602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1c3943ec-4f11-40f8-9aa9-f51bc3663f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:32.117110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 516/1028] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:20:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:36.106162 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3302aa-efef-4680-8870-892c176625f5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:36.106992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3302aa-efef-4680-8870-892c176625f5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:36.107421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3302aa-efef-4680-8870-892c176625f5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:36.108754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 513/1029] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:36.113831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaacb0-c0ba-4868-8fdb-a0cae21071da None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:36.114274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaacb0-c0ba-4868-8fdb-a0cae21071da None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:36.114562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaacb0-c0ba-4868-8fdb-a0cae21071da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:36.173509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:36.199447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eebaacb0-c0ba-4868-8fdb-a0cae21071da None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:36.200772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eebaacb0-c0ba-4868-8fdb-a0cae21071da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:36.241155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=59 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:36.316353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 517/1030] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:20:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:37.812097 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3e6c-d8d1-4659-bfd9-6dea4766cc2f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:37.812709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3e6c-d8d1-4659-bfd9-6dea4766cc2f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:37.812709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d3e6c-d8d1-4659-bfd9-6dea4766cc2f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:37.814140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 514/1031] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:37.819600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6fce-5fb2-4102-b129-860e0831cf91 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:37.820050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6fce-5fb2-4102-b129-860e0831cf91 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:37.820228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c6fce-5fb2-4102-b129-860e0831cf91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:37.901286 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7e1c6fce-5fb2-4102-b129-860e0831cf91 None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:37.902583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e1c6fce-5fb2-4102-b129-860e0831cf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:37.947651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:38.014080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 518/1032] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:20:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:41.546257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187002cb-558e-44dd-a7e3-af4068d76f8b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:41.547194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187002cb-558e-44dd-a7e3-af4068d76f8b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:41.547628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187002cb-558e-44dd-a7e3-af4068d76f8b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:41.549194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 515/1033] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:41.554697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cab1c-c7ca-4a5f-b157-da8851c1d7a8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:41.555210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cab1c-c7ca-4a5f-b157-da8851c1d7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:41.555301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cab1c-c7ca-4a5f-b157-da8851c1d7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:41.626938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ee2cab1c-c7ca-4a5f-b157-da8851c1d7a8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:41.628131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ee2cab1c-c7ca-4a5f-b157-da8851c1d7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:41.729060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 519/1034] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:42.029660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a7470-4d22-4ca3-a034-133cc58b6869 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:42.030647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a7470-4d22-4ca3-a034-133cc58b6869 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:42.031049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a7470-4d22-4ca3-a034-133cc58b6869 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:42.032323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 516/1035] 10.222.0.25 () {54 vars in 961 bytes} [Wed Nov 2 07:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:42.042484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8ad04-49c3-41ad-822c-b619e3d65566 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:42.043142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8ad04-49c3-41ad-822c-b619e3d65566 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:42.043945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8ad04-49c3-41ad-822c-b619e3d65566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:42.063755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:42.126509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ccf8ad04-49c3-41ad-822c-b619e3d65566 None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:42.127733 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ccf8ad04-49c3-41ad-822c-b619e3d65566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:42.246993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 520/1036] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Nov 2 07:20:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:42.255339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a7470-4d22-4ca3-a034-133cc58b6869 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:42.257068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-277a7470-4d22-4ca3-a034-133cc58b6869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:42.411569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a7470-4d22-4ca3-a034-133cc58b6869 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:42.487403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:42.535142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd635f-038b-46ca-8ab6-e96b394c2c16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:42.535871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd635f-038b-46ca-8ab6-e96b394c2c16 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:42.536271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd635f-038b-46ca-8ab6-e96b394c2c16 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:42.536637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd635f-038b-46ca-8ab6-e96b394c2c16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:42.538386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b1bd635f-038b-46ca-8ab6-e96b394c2c16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:42.640361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd635f-038b-46ca-8ab6-e96b394c2c16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:42.642872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd635f-038b-46ca-8ab6-e96b394c2c16 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:42.664054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 517/1037] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:45.023012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804efef-0d32-4ffc-95d1-f1de87a435b5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:45.023602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804efef-0d32-4ffc-95d1-f1de87a435b5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:45.023602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804efef-0d32-4ffc-95d1-f1de87a435b5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:45.024893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 521/1038] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:45.030531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f7888-dec7-4c28-ba65-35e3c1c5b33d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:45.030990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f7888-dec7-4c28-ba65-35e3c1c5b33d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:45.031336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f7888-dec7-4c28-ba65-35e3c1c5b33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:45.136200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b14f7888-dec7-4c28-ba65-35e3c1c5b33d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:45.137550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b14f7888-dec7-4c28-ba65-35e3c1c5b33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:45.179787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:45.244711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 518/1039] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:45.533735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804efef-0d32-4ffc-95d1-f1de87a435b5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:45.536720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d804efef-0d32-4ffc-95d1-f1de87a435b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:45.616300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:45.723173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804efef-0d32-4ffc-95d1-f1de87a435b5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:45.850648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39afb3-c980-493d-8f77-3c549f90de14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:45.850648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39afb3-c980-493d-8f77-3c549f90de14 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:45.850962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39afb3-c980-493d-8f77-3c549f90de14 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:45.850962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39afb3-c980-493d-8f77-3c549f90de14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:45.852594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6b39afb3-c980-493d-8f77-3c549f90de14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:45.877813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:45.950787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39afb3-c980-493d-8f77-3c549f90de14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:45.952387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39afb3-c980-493d-8f77-3c549f90de14 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:45.971237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 522/1040] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:48.465080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d265-3a7f-4162-9e26-0e908b96750a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:48.465626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d265-3a7f-4162-9e26-0e908b96750a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:48.465895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d265-3a7f-4162-9e26-0e908b96750a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:48.467653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 519/1041] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:48.473214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bac90-fec7-4a8a-8659-d1c0e7756c29 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:48.473686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bac90-fec7-4a8a-8659-d1c0e7756c29 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:48.473789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bac90-fec7-4a8a-8659-d1c0e7756c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:48.543208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cf1bac90-fec7-4a8a-8659-d1c0e7756c29 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:48.544304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cf1bac90-fec7-4a8a-8659-d1c0e7756c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:48.655531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:48.658599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 523/1042] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:48.948314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadfe2d-a85e-4cb9-a1cc-47454b664ad6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:48.948607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadfe2d-a85e-4cb9-a1cc-47454b664ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:48.948876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadfe2d-a85e-4cb9-a1cc-47454b664ad6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:48.950367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 520/1043] 10.222.0.25 () {54 vars in 961 bytes} [Wed Nov 2 07:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:48.956444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4511f-9409-456e-ba93-660742d762ea None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:48.956856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4511f-9409-456e-ba93-660742d762ea None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:48.957643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4511f-9409-456e-ba93-660742d762ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:49.047932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-44d4511f-9409-456e-ba93-660742d762ea None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:49.049343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-44d4511f-9409-456e-ba93-660742d762ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:49.173800 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 524/1044] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Nov 2 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:49.182115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadfe2d-a85e-4cb9-a1cc-47454b664ad6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:49.183730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4dadfe2d-a85e-4cb9-a1cc-47454b664ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:49.343268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadfe2d-a85e-4cb9-a1cc-47454b664ad6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:49.364721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:49.467469 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9fee2-f44b-4d18-aaa6-3e34988ff8f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:49.468137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9fee2-f44b-4d18-aaa6-3e34988ff8f5 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:49.468640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9fee2-f44b-4d18-aaa6-3e34988ff8f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:49.469001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9fee2-f44b-4d18-aaa6-3e34988ff8f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:49.470837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-70e9fee2-f44b-4d18-aaa6-3e34988ff8f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:49.581509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9fee2-f44b-4d18-aaa6-3e34988ff8f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:49.583450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9fee2-f44b-4d18-aaa6-3e34988ff8f5 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:49.604133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 521/1045] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:49.659226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311261d-61ce-4341-be4f-de141f71f715 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:49.659736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311261d-61ce-4341-be4f-de141f71f715 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:49.659856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311261d-61ce-4341-be4f-de141f71f715 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:49.661332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 525/1046] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:49.668080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f2f69-0eb3-4717-b67f-2e8149d3042c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:49.668367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f2f69-0eb3-4717-b67f-2e8149d3042c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:49.668670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f2f69-0eb3-4717-b67f-2e8149d3042c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:49.703071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:49.748936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1b6f2f69-0eb3-4717-b67f-2e8149d3042c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:49.750260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1b6f2f69-0eb3-4717-b67f-2e8149d3042c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:49.866924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 522/1047] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:50.201249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2e3d-3375-4d3d-960b-6279b5e30213 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:50.201459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2e3d-3375-4d3d-960b-6279b5e30213 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:50.201732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2e3d-3375-4d3d-960b-6279b5e30213 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:50.203130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 526/1048] 10.222.0.25 () {54 vars in 963 bytes} [Wed Nov 2 07:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:50.208232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbbd43-c62f-46be-b254-67c0fcb207e7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:50.208512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbbd43-c62f-46be-b254-67c0fcb207e7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:50.208786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbbd43-c62f-46be-b254-67c0fcb207e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:50.292438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f2fbbd43-c62f-46be-b254-67c0fcb207e7 None None] MFA Rules not processed for user `d17ba458c5004646b5e21cabcb54fd20`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:50.293789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f2fbbd43-c62f-46be-b254-67c0fcb207e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:50.341361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:50.375196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 523/1049] 10.222.0.25 () {58 vars in 1060 bytes} [Wed Nov 2 07:20:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:50.383555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c2e3d-3375-4d3d-960b-6279b5e30213 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:50.385114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-811c2e3d-3375-4d3d-960b-6279b5e30213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:50.445000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:50.507277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c2e3d-3375-4d3d-960b-6279b5e30213 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:50.596840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498adc87-573c-49af-bb14-ae6acce8300a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:50.597596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498adc87-573c-49af-bb14-ae6acce8300a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:50.597931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498adc87-573c-49af-bb14-ae6acce8300a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:50.598288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498adc87-573c-49af-bb14-ae6acce8300a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:50.600756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-498adc87-573c-49af-bb14-ae6acce8300a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:50.705599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:50.720334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498adc87-573c-49af-bb14-ae6acce8300a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:50.721810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498adc87-573c-49af-bb14-ae6acce8300a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:50.742363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 527/1050] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:51.838014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a3216-0b24-4530-9793-9c7adfaabfd2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:51.838983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a3216-0b24-4530-9793-9c7adfaabfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:51.839288 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a3216-0b24-4530-9793-9c7adfaabfd2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:51.841047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 524/1051] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:51.846954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfafb28-6f4f-4554-b7c3-fe68f1ff4ced None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:51.847423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfafb28-6f4f-4554-b7c3-fe68f1ff4ced None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:51.847535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfafb28-6f4f-4554-b7c3-fe68f1ff4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:51.922253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-acfafb28-6f4f-4554-b7c3-fe68f1ff4ced None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:51.923470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-acfafb28-6f4f-4554-b7c3-fe68f1ff4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:52.020049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 528/1052] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:20:52.312134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a3216-0b24-4530-9793-9c7adfaabfd2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:52.313809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-821a3216-0b24-4530-9793-9c7adfaabfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:52.471870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a3216-0b24-4530-9793-9c7adfaabfd2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:52.524404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:52.595075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04130b17-c71f-4e58-bdc7-9a8e7688f960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:52.595694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04130b17-c71f-4e58-bdc7-9a8e7688f960 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:52.595973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04130b17-c71f-4e58-bdc7-9a8e7688f960 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:52.596246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04130b17-c71f-4e58-bdc7-9a8e7688f960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:52.597916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04130b17-c71f-4e58-bdc7-9a8e7688f960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:52.691516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04130b17-c71f-4e58-bdc7-9a8e7688f960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:52.693258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04130b17-c71f-4e58-bdc7-9a8e7688f960 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:52.712900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 525/1053] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:54.932772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aad476-f406-40db-b564-a3615c333a1a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:54.933292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aad476-f406-40db-b564-a3615c333a1a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:54.933418 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aad476-f406-40db-b564-a3615c333a1a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:54.934882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 529/1054] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:54.940034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687a48a-3696-4da3-b727-c769b9cb08ce None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:54.940429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687a48a-3696-4da3-b727-c769b9cb08ce None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:54.940577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687a48a-3696-4da3-b727-c769b9cb08ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:55.009566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1687a48a-3696-4da3-b727-c769b9cb08ce None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:55.010767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1687a48a-3696-4da3-b727-c769b9cb08ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:55.018619 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:55.121703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 526/1055] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:55.409758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aad476-f406-40db-b564-a3615c333a1a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:55.411315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f1aad476-f406-40db-b564-a3615c333a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:55.477490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:55.565735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aad476-f406-40db-b564-a3615c333a1a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:55.685457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884add1b-cfc5-441c-85c7-a635e6f13487 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:55.685980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884add1b-cfc5-441c-85c7-a635e6f13487 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:55.686270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884add1b-cfc5-441c-85c7-a635e6f13487 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:55.686543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884add1b-cfc5-441c-85c7-a635e6f13487 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:55.688465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-884add1b-cfc5-441c-85c7-a635e6f13487 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:55.708579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:55.784134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884add1b-cfc5-441c-85c7-a635e6f13487 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:55.785725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884add1b-cfc5-441c-85c7-a635e6f13487 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:55.806060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 530/1056] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:58.026991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c4c63-9777-4e0c-be69-469b73592587 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:58.027452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c4c63-9777-4e0c-be69-469b73592587 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:58.027540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c4c63-9777-4e0c-be69-469b73592587 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:58.029052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 527/1057] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:58.034219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538d0e-1acf-4c2f-ac1b-bfd91c248a71 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:58.034618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538d0e-1acf-4c2f-ac1b-bfd91c248a71 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:58.034773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538d0e-1acf-4c2f-ac1b-bfd91c248a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:58.112217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-79538d0e-1acf-4c2f-ac1b-bfd91c248a71 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:58.113540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-79538d0e-1acf-4c2f-ac1b-bfd91c248a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.210716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:58.230842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 531/1058] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:58.530227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c4c63-9777-4e0c-be69-469b73592587 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:58.531881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-753c4c63-9777-4e0c-be69-469b73592587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.699444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:58.711428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c4c63-9777-4e0c-be69-469b73592587 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:58.848258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261c3bf-40cf-4cb8-acfb-cf49f10120bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:58.848923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261c3bf-40cf-4cb8-acfb-cf49f10120bb service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:58.849227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261c3bf-40cf-4cb8-acfb-cf49f10120bb service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:58.849499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261c3bf-40cf-4cb8-acfb-cf49f10120bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:58.851424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c261c3bf-40cf-4cb8-acfb-cf49f10120bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.975399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261c3bf-40cf-4cb8-acfb-cf49f10120bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:58.977601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261c3bf-40cf-4cb8-acfb-cf49f10120bb service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:58.989263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:58.998410 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 528/1059] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:01.275514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27eed2-1457-45e1-bceb-f2846084039e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:01.276052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27eed2-1457-45e1-bceb-f2846084039e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:01.276052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27eed2-1457-45e1-bceb-f2846084039e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:01.277554 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 532/1060] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:01.283599 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85048f-73d0-4950-957b-5654168d589a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:01.284024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85048f-73d0-4950-957b-5654168d589a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:01.284152 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85048f-73d0-4950-957b-5654168d589a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:01.357220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ae85048f-73d0-4950-957b-5654168d589a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:01.358482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ae85048f-73d0-4950-957b-5654168d589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:01.486143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 529/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:01.771064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27eed2-1457-45e1-bceb-f2846084039e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:01.773423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cd27eed2-1457-45e1-bceb-f2846084039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:01.943890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27eed2-1457-45e1-bceb-f2846084039e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:02.023106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:02.070393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47c4f1-9572-41c5-b2aa-8f307414c9c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:02.071017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47c4f1-9572-41c5-b2aa-8f307414c9c3 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:02.071273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47c4f1-9572-41c5-b2aa-8f307414c9c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:02.071543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47c4f1-9572-41c5-b2aa-8f307414c9c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:02.073178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2b47c4f1-9572-41c5-b2aa-8f307414c9c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:02.174077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47c4f1-9572-41c5-b2aa-8f307414c9c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:02.175858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47c4f1-9572-41c5-b2aa-8f307414c9c3 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:02.195702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 533/1062] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:04.447094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6208b5-fa67-48ef-950f-35cf16eea422 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:04.447974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6208b5-fa67-48ef-950f-35cf16eea422 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:04.447974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6208b5-fa67-48ef-950f-35cf16eea422 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:04.449637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 530/1063] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:04.455114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb102-247b-4a9c-966c-729e790accc5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:04.455579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb102-247b-4a9c-966c-729e790accc5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:04.455717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb102-247b-4a9c-966c-729e790accc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:04.524673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-deffb102-247b-4a9c-966c-729e790accc5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:04.525891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-deffb102-247b-4a9c-966c-729e790accc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:04.562371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:04.628932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 534/1064] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:04.932235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6208b5-fa67-48ef-950f-35cf16eea422 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:04.933801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1d6208b5-fa67-48ef-950f-35cf16eea422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:05.013625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:05.086835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6208b5-fa67-48ef-950f-35cf16eea422 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:05.208043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be273b9a-5a8b-497e-8f9e-d38e3dce170b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:05.208571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273b9a-5a8b-497e-8f9e-d38e3dce170b service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:05.209031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273b9a-5a8b-497e-8f9e-d38e3dce170b service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:05.209215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273b9a-5a8b-497e-8f9e-d38e3dce170b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:05.211230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-be273b9a-5a8b-497e-8f9e-d38e3dce170b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:05.249597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:05.302967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be273b9a-5a8b-497e-8f9e-d38e3dce170b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:05.304679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be273b9a-5a8b-497e-8f9e-d38e3dce170b service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:05.323415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 531/1065] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:05.980963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40c53-259c-474f-ad1f-445634c6c23c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:05.981547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40c53-259c-474f-ad1f-445634c6c23c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:05.981547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40c53-259c-474f-ad1f-445634c6c23c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:05.982890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 535/1066] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:05.988264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f28e45-c5a8-42cb-b5f2-12c6b11afee4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:05.988740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f28e45-c5a8-42cb-b5f2-12c6b11afee4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:05.988853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f28e45-c5a8-42cb-b5f2-12c6b11afee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:06.056284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e1f28e45-c5a8-42cb-b5f2-12c6b11afee4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:06.057481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e1f28e45-c5a8-42cb-b5f2-12c6b11afee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:06.158746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 532/1067] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:06.457750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9339-068e-4086-b9c4-042f2eba2bbf None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:06.458030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9339-068e-4086-b9c4-042f2eba2bbf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:06.458361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab9339-068e-4086-b9c4-042f2eba2bbf None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:06.459859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 536/1068] 10.222.0.25 () {54 vars in 962 bytes} [Wed Nov 2 07:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:06.465676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54542-9797-4427-ae7f-64da74a827ba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:06.465961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54542-9797-4427-ae7f-64da74a827ba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:06.466272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54542-9797-4427-ae7f-64da74a827ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:06.482411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:06.545563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f7b54542-9797-4427-ae7f-64da74a827ba None None] MFA Rules not processed for user `f9fd09151e284f639c93699b0d26db05`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:06.546793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f7b54542-9797-4427-ae7f-64da74a827ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:06.627229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 533/1069] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Nov 2 07:21:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:06.636435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab9339-068e-4086-b9c4-042f2eba2bbf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:06.638104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b7ab9339-068e-4086-b9c4-042f2eba2bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:06.750477 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab9339-068e-4086-b9c4-042f2eba2bbf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:06.813343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:06.828650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a491c-dc38-4750-a39c-569f227c6c71 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fd09151e284f639c93699b0d26db05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:06.829254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a491c-dc38-4750-a39c-569f227c6c71 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:06.829524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a491c-dc38-4750-a39c-569f227c6c71 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:06.829759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a491c-dc38-4750-a39c-569f227c6c71 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:06.831378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a02a491c-dc38-4750-a39c-569f227c6c71 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:06.948880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a491c-dc38-4750-a39c-569f227c6c71 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:06.950356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a491c-dc38-4750-a39c-569f227c6c71 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:06.969896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 537/1070] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:07.524967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87568b-8287-482b-a7bb-89bc47705339 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:07.525569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87568b-8287-482b-a7bb-89bc47705339 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:07.525569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87568b-8287-482b-a7bb-89bc47705339 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:07.526992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 534/1071] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:07.533172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663020bc-b9b9-416e-acca-81ff1bc52b91 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:07.533772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663020bc-b9b9-416e-acca-81ff1bc52b91 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:07.533772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663020bc-b9b9-416e-acca-81ff1bc52b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:07.615205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-663020bc-b9b9-416e-acca-81ff1bc52b91 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:07.616525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-663020bc-b9b9-416e-acca-81ff1bc52b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:07.662714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:07.727936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4a393-b363-4035-a6a7-e1067a740668 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:07.728222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4a393-b363-4035-a6a7-e1067a740668 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:07.728371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 538/1072] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:07.728484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4a393-b363-4035-a6a7-e1067a740668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:07.793915 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-97f4a393-b363-4035-a6a7-e1067a740668 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:07.795147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-97f4a393-b363-4035-a6a7-e1067a740668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:07.853806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:07.886297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 535/1073] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:21:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:07.930464 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be1497-5ebf-48a5-b80d-46f4ce38ebe6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:07.930822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be1497-5ebf-48a5-b80d-46f4ce38ebe6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:07.931185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be1497-5ebf-48a5-b80d-46f4ce38ebe6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:07.932606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 539/1074] 10.222.0.25 () {54 vars in 943 bytes} [Wed Nov 2 07:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:07.938926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d716-bb30-4f3a-bca7-a4bc54878b29 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:07.939275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d716-bb30-4f3a-bca7-a4bc54878b29 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:07.939610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d716-bb30-4f3a-bca7-a4bc54878b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:08.013220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be1497-5ebf-48a5-b80d-46f4ce38ebe6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:08.014933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-31be1497-5ebf-48a5-b80d-46f4ce38ebe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:08.020292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9ed1d716-bb30-4f3a-bca7-a4bc54878b29 None None] MFA Rules not processed for user `c3ab55070be14274857478b5581d2a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:08.021532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9ed1d716-bb30-4f3a-bca7-a4bc54878b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:08.128653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 536/1075] 10.222.0.25 () {58 vars in 1040 bytes} [Wed Nov 2 07:21:07 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:08.138164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1d716-bb30-4f3a-bca7-a4bc54878b29 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:08.147591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1d716-bb30-4f3a-bca7-a4bc54878b29 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:08.169138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be1497-5ebf-48a5-b80d-46f4ce38ebe6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:08.173235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:08.196867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:08.256477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6db22b-1280-40c2-803b-285cb96c8788 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:08.257017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6db22b-1280-40c2-803b-285cb96c8788 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:08.257291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6db22b-1280-40c2-803b-285cb96c8788 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:08.257558 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6db22b-1280-40c2-803b-285cb96c8788 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:08.259250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4c6db22b-1280-40c2-803b-285cb96c8788 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:08.285142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930436-a655-45b1-a4b0-e8377fd745d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:08.285750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930436-a655-45b1-a4b0-e8377fd745d6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:08.286022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930436-a655-45b1-a4b0-e8377fd745d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:08.286352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930436-a655-45b1-a4b0-e8377fd745d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:08.287951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e3930436-a655-45b1-a4b0-e8377fd745d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:08.343575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6db22b-1280-40c2-803b-285cb96c8788 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:08.345189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6db22b-1280-40c2-803b-285cb96c8788 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:08.353482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 537/1076] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:21:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:21:08.378866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3930436-a655-45b1-a4b0-e8377fd745d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:08.380474 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3930436-a655-45b1-a4b0-e8377fd745d6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:08.400504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 540/1077] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:10.605996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932c4a3-88ef-4ace-85d1-9feeae782bfd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:10.607110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932c4a3-88ef-4ace-85d1-9feeae782bfd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:10.607466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932c4a3-88ef-4ace-85d1-9feeae782bfd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:10.610123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 538/1078] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:10.614151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddffe34-a62e-487c-98ad-6288736bf10c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:10.614594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddffe34-a62e-487c-98ad-6288736bf10c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:10.614710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddffe34-a62e-487c-98ad-6288736bf10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:10.655984 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:10.691249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2ddffe34-a62e-487c-98ad-6288736bf10c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:10.692477 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2ddffe34-a62e-487c-98ad-6288736bf10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:10.799806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 541/1079] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:11.089953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932c4a3-88ef-4ace-85d1-9feeae782bfd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:11.091643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e932c4a3-88ef-4ace-85d1-9feeae782bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:11.160408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:11.254412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932c4a3-88ef-4ace-85d1-9feeae782bfd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:11.377434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d43bf-ff93-4776-8fa3-c304e9c9647e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:11.378169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d43bf-ff93-4776-8fa3-c304e9c9647e service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:11.378448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d43bf-ff93-4776-8fa3-c304e9c9647e service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:11.378720 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d43bf-ff93-4776-8fa3-c304e9c9647e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:11.380291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-939d43bf-ff93-4776-8fa3-c304e9c9647e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:11.400385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:11.485378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d43bf-ff93-4776-8fa3-c304e9c9647e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:11.487794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d43bf-ff93-4776-8fa3-c304e9c9647e service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:11.508910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 539/1080] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:13.734471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55737d-768b-4964-9f6b-1b1652a7a6f6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:13.735037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55737d-768b-4964-9f6b-1b1652a7a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:13.735037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55737d-768b-4964-9f6b-1b1652a7a6f6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:13.736290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 542/1081] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:13.742993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e383b1-05a3-4669-a666-3a02e56a5ce4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:13.746592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e383b1-05a3-4669-a666-3a02e56a5ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:13.746592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e383b1-05a3-4669-a666-3a02e56a5ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:13.830566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-71e383b1-05a3-4669-a666-3a02e56a5ce4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:13.832189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-71e383b1-05a3-4669-a666-3a02e56a5ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:13.934864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:13.942758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 540/1082] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:14.234132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55737d-768b-4964-9f6b-1b1652a7a6f6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:14.236302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f55737d-768b-4964-9f6b-1b1652a7a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:14.357853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:14.402155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55737d-768b-4964-9f6b-1b1652a7a6f6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:14.527771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dbfa3-5ffe-4616-b06f-f3b0fac67969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:14.528287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dbfa3-5ffe-4616-b06f-f3b0fac67969 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:14.528556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dbfa3-5ffe-4616-b06f-f3b0fac67969 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:14.528823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dbfa3-5ffe-4616-b06f-f3b0fac67969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:14.530475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-536dbfa3-5ffe-4616-b06f-f3b0fac67969 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:14.610647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:14.635924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536dbfa3-5ffe-4616-b06f-f3b0fac67969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:14.637529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536dbfa3-5ffe-4616-b06f-f3b0fac67969 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:14.660275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 543/1083] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:16.924586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d2ea-a88f-405b-8797-20fa29a25b39 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.925099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d2ea-a88f-405b-8797-20fa29a25b39 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.925585 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d2ea-a88f-405b-8797-20fa29a25b39 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:16.928455 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 541/1084] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:21:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:16.935510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fedf-f3ad-4a7a-85bb-d13fbe05fca5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.936421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fedf-f3ad-4a7a-85bb-d13fbe05fca5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.936903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fedf-f3ad-4a7a-85bb-d13fbe05fca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:17.038604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22a2fedf-f3ad-4a7a-85bb-d13fbe05fca5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:17.040225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22a2fedf-f3ad-4a7a-85bb-d13fbe05fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:17.166936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 544/1085] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:17.466383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9d2ea-a88f-405b-8797-20fa29a25b39 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:17.467933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dfc9d2ea-a88f-405b-8797-20fa29a25b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:17.641802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9d2ea-a88f-405b-8797-20fa29a25b39 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:17.723923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:17.772902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c3bbf-1fd3-492a-99a8-b6eec356aa36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:17.773788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c3bbf-1fd3-492a-99a8-b6eec356aa36 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:17.774277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c3bbf-1fd3-492a-99a8-b6eec356aa36 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:17.774695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c3bbf-1fd3-492a-99a8-b6eec356aa36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:17.776596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e13c3bbf-1fd3-492a-99a8-b6eec356aa36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:17.890129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c3bbf-1fd3-492a-99a8-b6eec356aa36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:17.891762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c3bbf-1fd3-492a-99a8-b6eec356aa36 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:17.921071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 542/1086] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:21:27.165451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=79 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:27.918375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=64 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:30.944075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84bfd7-d3ce-49b8-b823-0678fa5c68a8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:30.944626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84bfd7-d3ce-49b8-b823-0678fa5c68a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:30.944626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84bfd7-d3ce-49b8-b823-0678fa5c68a8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:30.945961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 545/1087] 10.222.0.25 () {54 vars in 918 bytes} [Wed Nov 2 07:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:30.951421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33b017-5978-4ee8-9a62-98d8c3ba4006 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:30.951874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33b017-5978-4ee8-9a62-98d8c3ba4006 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:30.951987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33b017-5978-4ee8-9a62-98d8c3ba4006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:31.032526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cd33b017-5978-4ee8-9a62-98d8c3ba4006 None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:31.033726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cd33b017-5978-4ee8-9a62-98d8c3ba4006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:31.143644 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 543/1088] 10.222.0.25 () {58 vars in 1015 bytes} [Wed Nov 2 07:21:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:31.163696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259f4f-ca48-48c6-ae88-6113306e3199 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:31.163974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259f4f-ca48-48c6-ae88-6113306e3199 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:31.164240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259f4f-ca48-48c6-ae88-6113306e3199 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:31.165613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 546/1089] 10.222.0.25 () {54 vars in 918 bytes} [Wed Nov 2 07:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:31.170558 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e870c1-181c-4262-87a2-c640bf197083 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:31.170777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e870c1-181c-4262-87a2-c640bf197083 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:31.171051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e870c1-181c-4262-87a2-c640bf197083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:31.256268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:31.259587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-44e870c1-181c-4262-87a2-c640bf197083 None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:31.261259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-44e870c1-181c-4262-87a2-c640bf197083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:31.388031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 544/1090] 10.222.0.25 () {58 vars in 1015 bytes} [Wed Nov 2 07:21:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:33.986676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732f048-7ef9-4a03-84c3-9ee3aaa9a460 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:33.987271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732f048-7ef9-4a03-84c3-9ee3aaa9a460 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:33.987271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732f048-7ef9-4a03-84c3-9ee3aaa9a460 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:33.988594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 547/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:33.995074 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3fa28-b730-4077-aba1-1276369e3c0e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:33.995851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3fa28-b730-4077-aba1-1276369e3c0e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:33.995992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3fa28-b730-4077-aba1-1276369e3c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:34.068261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7ed3fa28-b730-4077-aba1-1276369e3c0e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:34.069466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7ed3fa28-b730-4077-aba1-1276369e3c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:34.126529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:34.171356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 545/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:34.464611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732f048-7ef9-4a03-84c3-9ee3aaa9a460 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:34.466385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a732f048-7ef9-4a03-84c3-9ee3aaa9a460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:34.622994 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732f048-7ef9-4a03-84c3-9ee3aaa9a460 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:34.700448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:34.744612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ce061-0aa6-4cfc-b0b2-992c1f72e451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:34.745226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ce061-0aa6-4cfc-b0b2-992c1f72e451 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:34.745525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ce061-0aa6-4cfc-b0b2-992c1f72e451 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:34.745796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ce061-0aa6-4cfc-b0b2-992c1f72e451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:34.747448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac9ce061-0aa6-4cfc-b0b2-992c1f72e451 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:34.842959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ce061-0aa6-4cfc-b0b2-992c1f72e451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:34.844542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ce061-0aa6-4cfc-b0b2-992c1f72e451 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:34.863002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 548/1093] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:35.996934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65704abf-7713-4f7b-ace5-0638a62ec209 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:35.997533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65704abf-7713-4f7b-ace5-0638a62ec209 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:35.997533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65704abf-7713-4f7b-ace5-0638a62ec209 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:35.998889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 546/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:36.004772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0460-ec9e-4dd1-bb68-1542cc689777 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.005263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0460-ec9e-4dd1-bb68-1542cc689777 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.005370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0460-ec9e-4dd1-bb68-1542cc689777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.092719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-922d0460-ec9e-4dd1-bb68-1542cc689777 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:36.093932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-922d0460-ec9e-4dd1-bb68-1542cc689777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:36.121919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:36.193223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 549/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:36.356921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65704abf-7713-4f7b-ace5-0638a62ec209 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:36.359413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-65704abf-7713-4f7b-ace5-0638a62ec209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:36.524221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65704abf-7713-4f7b-ace5-0638a62ec209 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:36.568205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:36.644217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3104a0-60bc-47a1-8b4d-8f05135d9aab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:36.644842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3104a0-60bc-47a1-8b4d-8f05135d9aab service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.645112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3104a0-60bc-47a1-8b4d-8f05135d9aab service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.645376 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3104a0-60bc-47a1-8b4d-8f05135d9aab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.647462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c3104a0-60bc-47a1-8b4d-8f05135d9aab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:36.747553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3104a0-60bc-47a1-8b4d-8f05135d9aab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:36.749184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3104a0-60bc-47a1-8b4d-8f05135d9aab service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:36.768020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 547/1096] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:37.935545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e121b5-3dcb-477a-9ae8-edc18cae5f0e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:37.936061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e121b5-3dcb-477a-9ae8-edc18cae5f0e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:37.936061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e121b5-3dcb-477a-9ae8-edc18cae5f0e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:37.937438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 550/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:37.942692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1232773-3754-4583-8785-5c5fb2eb24f3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:37.943732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1232773-3754-4583-8785-5c5fb2eb24f3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:37.943732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1232773-3754-4583-8785-5c5fb2eb24f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:38.010363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:38.034608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a1232773-3754-4583-8785-5c5fb2eb24f3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:38.035796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a1232773-3754-4583-8785-5c5fb2eb24f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:38.144369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 548/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:38.297391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e121b5-3dcb-477a-9ae8-edc18cae5f0e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:38.299935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-78e121b5-3dcb-477a-9ae8-edc18cae5f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:38.466434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e121b5-3dcb-477a-9ae8-edc18cae5f0e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:38.479356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:38.587085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf7213-8ba7-4e03-ba24-a7331ac860dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:38.587761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf7213-8ba7-4e03-ba24-a7331ac860dd service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:38.588081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf7213-8ba7-4e03-ba24-a7331ac860dd service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:38.588377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf7213-8ba7-4e03-ba24-a7331ac860dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:38.590020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bdbf7213-8ba7-4e03-ba24-a7331ac860dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:38.682357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf7213-8ba7-4e03-ba24-a7331ac860dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:38.684805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf7213-8ba7-4e03-ba24-a7331ac860dd service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:38.708846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 551/1099] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:39.819059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db2f6-d07a-488f-aead-7e032b3cabe6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:39.819679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db2f6-d07a-488f-aead-7e032b3cabe6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:39.819679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db2f6-d07a-488f-aead-7e032b3cabe6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:39.820946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 549/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:39.827113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88795ba5-f105-4c24-b39b-5802088a8bd7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:39.827586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88795ba5-f105-4c24-b39b-5802088a8bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:39.827908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88795ba5-f105-4c24-b39b-5802088a8bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:39.857358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:39.917068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-88795ba5-f105-4c24-b39b-5802088a8bd7 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:39.918350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88795ba5-f105-4c24-b39b-5802088a8bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:40.028931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 552/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:40.179396 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db2f6-d07a-488f-aead-7e032b3cabe6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:40.182063 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a17db2f6-d07a-488f-aead-7e032b3cabe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:40.301742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:40.340024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db2f6-d07a-488f-aead-7e032b3cabe6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:40.463470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4d1e0-0aaf-42d9-9062-7435d339c55d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:40.463989 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4d1e0-0aaf-42d9-9062-7435d339c55d service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:40.464268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4d1e0-0aaf-42d9-9062-7435d339c55d service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:40.464531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4d1e0-0aaf-42d9-9062-7435d339c55d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:40.466138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f2b4d1e0-0aaf-42d9-9062-7435d339c55d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:40.546127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:40.560663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4d1e0-0aaf-42d9-9062-7435d339c55d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:40.562293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4d1e0-0aaf-42d9-9062-7435d339c55d service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:40.581344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 550/1102] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:41.681100 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9a95-99d5-4c1a-b06d-8270584524cb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:41.681977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9a95-99d5-4c1a-b06d-8270584524cb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:41.681977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9a95-99d5-4c1a-b06d-8270584524cb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:41.684263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 553/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:41.692432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f04b4-eb53-43ea-a174-6e80d2952858 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:41.693240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f04b4-eb53-43ea-a174-6e80d2952858 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:41.693411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f04b4-eb53-43ea-a174-6e80d2952858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:41.773038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f09f04b4-eb53-43ea-a174-6e80d2952858 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:41.774142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f09f04b4-eb53-43ea-a174-6e80d2952858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:41.876137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 551/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:42.029174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9a95-99d5-4c1a-b06d-8270584524cb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:42.030726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-23bf9a95-99d5-4c1a-b06d-8270584524cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:42.129395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:42.195139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9a95-99d5-4c1a-b06d-8270584524cb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:42.327522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1361f-dbc0-4df4-8fb3-fed7d1ded5cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:42.328050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1361f-dbc0-4df4-8fb3-fed7d1ded5cb service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:42.328327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1361f-dbc0-4df4-8fb3-fed7d1ded5cb service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:42.328594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1361f-dbc0-4df4-8fb3-fed7d1ded5cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:42.330238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5dc1361f-dbc0-4df4-8fb3-fed7d1ded5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:42.382172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:42.425871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1361f-dbc0-4df4-8fb3-fed7d1ded5cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:42.427561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1361f-dbc0-4df4-8fb3-fed7d1ded5cb service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:42.446628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 554/1105] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:43.484451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc091f3-d68e-474d-a722-0081669f7d7d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:43.485095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc091f3-d68e-474d-a722-0081669f7d7d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:43.485095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc091f3-d68e-474d-a722-0081669f7d7d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:43.487102 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 552/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:43.493559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42dca2-45b2-4eb9-a172-45ebc4a0b5cd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:43.493961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42dca2-45b2-4eb9-a172-45ebc4a0b5cd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:43.494170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42dca2-45b2-4eb9-a172-45ebc4a0b5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:43.575809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ff42dca2-45b2-4eb9-a172-45ebc4a0b5cd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:43.577027 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ff42dca2-45b2-4eb9-a172-45ebc4a0b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:43.677510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 555/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:43.840132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc091f3-d68e-474d-a722-0081669f7d7d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:43.842517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cc091f3-d68e-474d-a722-0081669f7d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:43.898737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:44.004892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc091f3-d68e-474d-a722-0081669f7d7d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:44.127644 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b282e-5038-48c0-8158-4bacca8ec895 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:44.128285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b282e-5038-48c0-8158-4bacca8ec895 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:44.128556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b282e-5038-48c0-8158-4bacca8ec895 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:44.128821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b282e-5038-48c0-8158-4bacca8ec895 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:44.130437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7b3b282e-5038-48c0-8158-4bacca8ec895 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:44.137416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:44.226486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b282e-5038-48c0-8158-4bacca8ec895 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:44.228103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b282e-5038-48c0-8158-4bacca8ec895 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:44.248693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 553/1108] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:45.281487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fe45e-1553-472b-9bf1-b757c8efd771 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.282124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fe45e-1553-472b-9bf1-b757c8efd771 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.282124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fe45e-1553-472b-9bf1-b757c8efd771 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.283308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 556/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:45.289809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849128-8105-41ac-9753-e133700b9163 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.290670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849128-8105-41ac-9753-e133700b9163 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.290670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849128-8105-41ac-9753-e133700b9163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.380545 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f9849128-8105-41ac-9753-e133700b9163 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:45.381766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f9849128-8105-41ac-9753-e133700b9163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:45.452546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:45.484261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 554/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:45.637636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fe45e-1553-472b-9bf1-b757c8efd771 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:45.639187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fe6fe45e-1553-472b-9bf1-b757c8efd771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:45.657384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:45.804710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fe45e-1553-472b-9bf1-b757c8efd771 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:45.896542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:45.924183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ab3f4-972d-441a-bb98-f1b48b773a0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:45.924775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab3f4-972d-441a-bb98-f1b48b773a0f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.925074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab3f4-972d-441a-bb98-f1b48b773a0f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.925344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab3f4-972d-441a-bb98-f1b48b773a0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.926968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd4ab3f4-972d-441a-bb98-f1b48b773a0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:46.022030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ab3f4-972d-441a-bb98-f1b48b773a0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:46.023653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ab3f4-972d-441a-bb98-f1b48b773a0f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:46.041873 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 557/1111] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:47.090846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae20b9-a7f8-4cbe-85b4-db6747774886 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:47.099339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae20b9-a7f8-4cbe-85b4-db6747774886 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:47.099339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae20b9-a7f8-4cbe-85b4-db6747774886 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:47.099339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 555/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:47.102160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768720-dd42-493e-9a91-614aab46bf5e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:47.102923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768720-dd42-493e-9a91-614aab46bf5e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:47.103086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768720-dd42-493e-9a91-614aab46bf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:47.190121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-48768720-dd42-493e-9a91-614aab46bf5e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:47.191245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-48768720-dd42-493e-9a91-614aab46bf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:47.238211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:47.298699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 558/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:47.449208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae20b9-a7f8-4cbe-85b4-db6747774886 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:47.451330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d5ae20b9-a7f8-4cbe-85b4-db6747774886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:47.609739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae20b9-a7f8-4cbe-85b4-db6747774886 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:47.670811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:47.733270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb9520-e181-437a-a584-c3c697ac4186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:47.733833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb9520-e181-437a-a584-c3c697ac4186 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:47.734167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb9520-e181-437a-a584-c3c697ac4186 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:47.734482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb9520-e181-437a-a584-c3c697ac4186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:47.736079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1cbb9520-e181-437a-a584-c3c697ac4186 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:47.829337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb9520-e181-437a-a584-c3c697ac4186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:47.831017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb9520-e181-437a-a584-c3c697ac4186 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:47.849584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 556/1114] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:48.883194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0020f50-7255-4966-af48-a75d3ace086c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:48.883829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0020f50-7255-4966-af48-a75d3ace086c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:48.883829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0020f50-7255-4966-af48-a75d3ace086c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:48.885095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 559/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:48.891138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29bf78-2cdd-4944-a683-4be418114889 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:48.891927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29bf78-2cdd-4944-a683-4be418114889 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:48.892074 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29bf78-2cdd-4944-a683-4be418114889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:48.987596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6a29bf78-2cdd-4944-a683-4be418114889 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:48.988820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a29bf78-2cdd-4944-a683-4be418114889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:48.990855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:49.093863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 557/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:49.244148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0020f50-7255-4966-af48-a75d3ace086c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:49.245567 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e0020f50-7255-4966-af48-a75d3ace086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:49.400549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0020f50-7255-4966-af48-a75d3ace086c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:49.428076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:49.525900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be22fd-2ddf-4413-a149-6a5bb5c12914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:49.526442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be22fd-2ddf-4413-a149-6a5bb5c12914 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:49.526712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be22fd-2ddf-4413-a149-6a5bb5c12914 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:49.526977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be22fd-2ddf-4413-a149-6a5bb5c12914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:49.528557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-42be22fd-2ddf-4413-a149-6a5bb5c12914 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:49.625521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be22fd-2ddf-4413-a149-6a5bb5c12914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:49.627178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be22fd-2ddf-4413-a149-6a5bb5c12914 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:49.646441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 560/1117] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:21:51.698306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901bfab-5fa2-4917-a4c4-37c5d1f5497c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:51.699005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901bfab-5fa2-4917-a4c4-37c5d1f5497c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:51.699005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901bfab-5fa2-4917-a4c4-37c5d1f5497c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:51.700197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 558/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:51.705712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786f3a-a7ac-488c-829d-b99d27f73630 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:51.706520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786f3a-a7ac-488c-829d-b99d27f73630 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:51.706960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786f3a-a7ac-488c-829d-b99d27f73630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:51.751289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:51.777133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-01786f3a-a7ac-488c-829d-b99d27f73630 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:51.778384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-01786f3a-a7ac-488c-829d-b99d27f73630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:51.889255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 561/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:52.171896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901bfab-5fa2-4917-a4c4-37c5d1f5497c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:52.174420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d901bfab-5fa2-4917-a4c4-37c5d1f5497c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:52.294058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901bfab-5fa2-4917-a4c4-37c5d1f5497c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:52.329080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:52.375003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54096d6-594c-4070-9cc5-cc8e1fd2f6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:52.375631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54096d6-594c-4070-9cc5-cc8e1fd2f6cf service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:52.375927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54096d6-594c-4070-9cc5-cc8e1fd2f6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:52.376190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54096d6-594c-4070-9cc5-cc8e1fd2f6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:52.377782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e54096d6-594c-4070-9cc5-cc8e1fd2f6cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:52.505987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54096d6-594c-4070-9cc5-cc8e1fd2f6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:52.507380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54096d6-594c-4070-9cc5-cc8e1fd2f6cf service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:52.525622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 559/1120] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:54.827266 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc526a7d-d930-43d6-93a2-6bb6720f423b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:54.827882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc526a7d-d930-43d6-93a2-6bb6720f423b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:54.827882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc526a7d-d930-43d6-93a2-6bb6720f423b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:54.829137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 562/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:54.836039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bee4c-b2ea-4ff5-9a9b-a2eb28dd705c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:54.836473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bee4c-b2ea-4ff5-9a9b-a2eb28dd705c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:54.836574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bee4c-b2ea-4ff5-9a9b-a2eb28dd705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:54.909331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-780bee4c-b2ea-4ff5-9a9b-a2eb28dd705c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:54.910585 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-780bee4c-b2ea-4ff5-9a9b-a2eb28dd705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:54.925363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:55.022078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 560/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:55.308038 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc526a7d-d930-43d6-93a2-6bb6720f423b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:55.309586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc526a7d-d930-43d6-93a2-6bb6720f423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:55.425442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc526a7d-d930-43d6-93a2-6bb6720f423b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:55.438067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:55.500809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bbcc5-8062-4a8f-81f1-5c81b277a67c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:55.501408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbcc5-8062-4a8f-81f1-5c81b277a67c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:55.501682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbcc5-8062-4a8f-81f1-5c81b277a67c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:55.501947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbcc5-8062-4a8f-81f1-5c81b277a67c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:55.503582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-940bbcc5-8062-4a8f-81f1-5c81b277a67c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:55.626898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bbcc5-8062-4a8f-81f1-5c81b277a67c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:55.628342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bbcc5-8062-4a8f-81f1-5c81b277a67c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:55.648274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 563/1123] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:58.124273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7717039-17ca-4ff7-92f6-69fc042ba644 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.124871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7717039-17ca-4ff7-92f6-69fc042ba644 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.124871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7717039-17ca-4ff7-92f6-69fc042ba644 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.126328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 561/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:58.134248 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c82c9-ead4-4a5e-b6d3-242a0dfec48f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.134690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c82c9-ead4-4a5e-b6d3-242a0dfec48f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.134781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c82c9-ead4-4a5e-b6d3-242a0dfec48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.186631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:58.208131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-667c82c9-ead4-4a5e-b6d3-242a0dfec48f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:58.209872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-667c82c9-ead4-4a5e-b6d3-242a0dfec48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.318877 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 564/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:58.603327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7717039-17ca-4ff7-92f6-69fc042ba644 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:58.605751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b7717039-17ca-4ff7-92f6-69fc042ba644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.726314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7717039-17ca-4ff7-92f6-69fc042ba644 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:58.769042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:58.807196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377d96d-add6-4173-9cec-65672ad152ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:58.807804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377d96d-add6-4173-9cec-65672ad152ba service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.808081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377d96d-add6-4173-9cec-65672ad152ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.808359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377d96d-add6-4173-9cec-65672ad152ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.809936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a377d96d-add6-4173-9cec-65672ad152ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.935485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377d96d-add6-4173-9cec-65672ad152ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:58.936939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377d96d-add6-4173-9cec-65672ad152ba service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:58.955385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 562/1126] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:01.292101 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a1b57-8417-479c-9330-52d581ec6d79 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:01.292679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a1b57-8417-479c-9330-52d581ec6d79 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:01.292679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a1b57-8417-479c-9330-52d581ec6d79 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:01.293918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 565/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:01.302220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e42102-8dc3-47ac-a272-eb35c1042ad5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:01.302689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e42102-8dc3-47ac-a272-eb35c1042ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:01.302800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e42102-8dc3-47ac-a272-eb35c1042ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:01.372858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d2e42102-8dc3-47ac-a272-eb35c1042ad5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:01.374079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d2e42102-8dc3-47ac-a272-eb35c1042ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:01.394626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:01.477170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 563/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:22:01.783231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a1b57-8417-479c-9330-52d581ec6d79 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:01.785753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-063a1b57-8417-479c-9330-52d581ec6d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:01.894650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a1b57-8417-479c-9330-52d581ec6d79 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:01.918532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:01.985008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28185a6-42d9-4264-9c39-dd8f1f55138e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:01.985614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28185a6-42d9-4264-9c39-dd8f1f55138e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:01.985886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28185a6-42d9-4264-9c39-dd8f1f55138e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:01.986179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28185a6-42d9-4264-9c39-dd8f1f55138e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:01.987772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d28185a6-42d9-4264-9c39-dd8f1f55138e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:02.108225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28185a6-42d9-4264-9c39-dd8f1f55138e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:02.109576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28185a6-42d9-4264-9c39-dd8f1f55138e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:02.127763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 566/1129] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:04.355112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e2b0-8250-4d62-aa6b-e93910b25870 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:04.355621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e2b0-8250-4d62-aa6b-e93910b25870 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:04.355621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e2b0-8250-4d62-aa6b-e93910b25870 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:04.356996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 564/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:04.362752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbf1d7-8309-4743-80ad-feef171d119c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:04.363190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbf1d7-8309-4743-80ad-feef171d119c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:04.363313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbf1d7-8309-4743-80ad-feef171d119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:04.428905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:04.432722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aadbf1d7-8309-4743-80ad-feef171d119c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:04.433931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aadbf1d7-8309-4743-80ad-feef171d119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:04.539409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 567/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:04.822506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9e2b0-8250-4d62-aa6b-e93910b25870 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:04.824047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7bf9e2b0-8250-4d62-aa6b-e93910b25870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:04.936774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9e2b0-8250-4d62-aa6b-e93910b25870 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:04.989764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:05.016968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa64d1-bf5b-4c3d-87df-e95edfc571b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:05.017578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa64d1-bf5b-4c3d-87df-e95edfc571b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:05.017849 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa64d1-bf5b-4c3d-87df-e95edfc571b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:05.018127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa64d1-bf5b-4c3d-87df-e95edfc571b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:05.019739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-25aa64d1-bf5b-4c3d-87df-e95edfc571b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:05.133073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa64d1-bf5b-4c3d-87df-e95edfc571b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:05.134577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa64d1-bf5b-4c3d-87df-e95edfc571b6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:05.152591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 565/1132] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:07.344802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45ca09-2653-4b37-b0cd-f37cab3d857b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:07.345632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45ca09-2653-4b37-b0cd-f37cab3d857b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:07.345632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45ca09-2653-4b37-b0cd-f37cab3d857b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:07.347486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 568/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:07.356552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e088c-ca18-4dd6-9e5d-5dc3722d9e70 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:07.357297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e088c-ca18-4dd6-9e5d-5dc3722d9e70 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:07.357630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e088c-ca18-4dd6-9e5d-5dc3722d9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:07.428688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-773e088c-ca18-4dd6-9e5d-5dc3722d9e70 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:07.430258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-773e088c-ca18-4dd6-9e5d-5dc3722d9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:07.458418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:07.534429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 566/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:07.818927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45ca09-2653-4b37-b0cd-f37cab3d857b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:07.820655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0e45ca09-2653-4b37-b0cd-f37cab3d857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:07.931253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45ca09-2653-4b37-b0cd-f37cab3d857b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:07.960303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:08.006385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddd2f8-3c20-4285-b1ce-19370c3ab669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:08.006988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd2f8-3c20-4285-b1ce-19370c3ab669 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:08.007274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd2f8-3c20-4285-b1ce-19370c3ab669 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:08.007552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd2f8-3c20-4285-b1ce-19370c3ab669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:08.009129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ceddd2f8-3c20-4285-b1ce-19370c3ab669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:08.119247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddd2f8-3c20-4285-b1ce-19370c3ab669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:08.120663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddd2f8-3c20-4285-b1ce-19370c3ab669 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:08.139428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 569/1135] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:10.458642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5575a36-d03e-4160-aa6c-dfd9d0f730d6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:10.459573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5575a36-d03e-4160-aa6c-dfd9d0f730d6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:10.459573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5575a36-d03e-4160-aa6c-dfd9d0f730d6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:10.461957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 567/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:10.468917 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04f46e-e45f-4f69-b6df-b238a8d2d029 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:10.469392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04f46e-e45f-4f69-b6df-b238a8d2d029 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:10.469486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04f46e-e45f-4f69-b6df-b238a8d2d029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:10.544750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fe04f46e-e45f-4f69-b6df-b238a8d2d029 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:10.545974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fe04f46e-e45f-4f69-b6df-b238a8d2d029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:10.554664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:10.650793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 570/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:10.931406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5575a36-d03e-4160-aa6c-dfd9d0f730d6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:10.933001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f5575a36-d03e-4160-aa6c-dfd9d0f730d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:11.053717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5575a36-d03e-4160-aa6c-dfd9d0f730d6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:11.111009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:11.133188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294d649-1c4f-49a6-b79b-3957066f49e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:11.133803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d649-1c4f-49a6-b79b-3957066f49e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:11.134080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d649-1c4f-49a6-b79b-3957066f49e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:11.134386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d649-1c4f-49a6-b79b-3957066f49e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:11.136020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e294d649-1c4f-49a6-b79b-3957066f49e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:11.267401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294d649-1c4f-49a6-b79b-3957066f49e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:11.268735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294d649-1c4f-49a6-b79b-3957066f49e6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:11.287805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 568/1138] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:13.494883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2b36-43fa-42da-846a-6aa76cef95f4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:13.495498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2b36-43fa-42da-846a-6aa76cef95f4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:13.495498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2b36-43fa-42da-846a-6aa76cef95f4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:13.496778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 571/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:13.503450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf01544-db62-478a-8860-fd8db3ae75b8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:13.503931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf01544-db62-478a-8860-fd8db3ae75b8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:13.504220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf01544-db62-478a-8860-fd8db3ae75b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:13.581703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dcf01544-db62-478a-8860-fd8db3ae75b8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:13.582972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dcf01544-db62-478a-8860-fd8db3ae75b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:13.623311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:13.684126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 569/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:13.968069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2b36-43fa-42da-846a-6aa76cef95f4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:13.969813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-343c2b36-43fa-42da-846a-6aa76cef95f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:14.077291 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2b36-43fa-42da-846a-6aa76cef95f4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:14.118758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:14.160282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb591c-ac05-4ee0-ac46-c75523c34a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:14.160912 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb591c-ac05-4ee0-ac46-c75523c34a13 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:14.161238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb591c-ac05-4ee0-ac46-c75523c34a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:14.161511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb591c-ac05-4ee0-ac46-c75523c34a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:14.163135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f6eb591c-ac05-4ee0-ac46-c75523c34a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:14.278823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb591c-ac05-4ee0-ac46-c75523c34a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:14.280387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb591c-ac05-4ee0-ac46-c75523c34a13 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:14.298804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 572/1141] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:16.494900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a5c45-53c7-4a7c-b654-7078d8f0e8ed None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:16.495521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a5c45-53c7-4a7c-b654-7078d8f0e8ed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:16.495521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a5c45-53c7-4a7c-b654-7078d8f0e8ed None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:16.496775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 570/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:16.503358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b4b43-16d7-405f-bfcf-32289366f81f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:16.503809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b4b43-16d7-405f-bfcf-32289366f81f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:16.503996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b4b43-16d7-405f-bfcf-32289366f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:16.579533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-188b4b43-16d7-405f-bfcf-32289366f81f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:16.580748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-188b4b43-16d7-405f-bfcf-32289366f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:16.602425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:16.687124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 573/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:16.971340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a5c45-53c7-4a7c-b654-7078d8f0e8ed None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:16.972888 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f50a5c45-53c7-4a7c-b654-7078d8f0e8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:17.082028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a5c45-53c7-4a7c-b654-7078d8f0e8ed None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:17.156317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:17.164310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21770e-96fa-43c1-87e4-c09cf24cf818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:17.164922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21770e-96fa-43c1-87e4-c09cf24cf818 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:17.165198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21770e-96fa-43c1-87e4-c09cf24cf818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:17.165468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21770e-96fa-43c1-87e4-c09cf24cf818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:17.167193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cb21770e-96fa-43c1-87e4-c09cf24cf818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:17.294531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21770e-96fa-43c1-87e4-c09cf24cf818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:17.295983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21770e-96fa-43c1-87e4-c09cf24cf818 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:17.317000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 571/1144] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:22:19.591179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef5fe1-1828-47c3-baf6-0d1e6a0501e6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:19.591756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef5fe1-1828-47c3-baf6-0d1e6a0501e6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:19.591756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef5fe1-1828-47c3-baf6-0d1e6a0501e6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:19.593051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 574/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:19.599531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750837b9-25f1-4f6a-b55b-a9bc4999d2ef None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:19.600259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750837b9-25f1-4f6a-b55b-a9bc4999d2ef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:19.600403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750837b9-25f1-4f6a-b55b-a9bc4999d2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:19.667259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-750837b9-25f1-4f6a-b55b-a9bc4999d2ef None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:19.668447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-750837b9-25f1-4f6a-b55b-a9bc4999d2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:19.719917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:19.776353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 572/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:20.061856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef5fe1-1828-47c3-baf6-0d1e6a0501e6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:20.064323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f2ef5fe1-1828-47c3-baf6-0d1e6a0501e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:20.189724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef5fe1-1828-47c3-baf6-0d1e6a0501e6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:20.243927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:20.270131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e26f2-ad7e-4683-a356-e96b15e13413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:20.270731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e26f2-ad7e-4683-a356-e96b15e13413 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:20.271000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e26f2-ad7e-4683-a356-e96b15e13413 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:20.271263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e26f2-ad7e-4683-a356-e96b15e13413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:20.272899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-687e26f2-ad7e-4683-a356-e96b15e13413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:20.388355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e26f2-ad7e-4683-a356-e96b15e13413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:20.389817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e26f2-ad7e-4683-a356-e96b15e13413 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:20.409759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 575/1147] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:29.773708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=22 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:30.407228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=66 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:40.450598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d240b-f222-4ad3-83fa-142065b7d647 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:40.451242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d240b-f222-4ad3-83fa-142065b7d647 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:40.451242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d240b-f222-4ad3-83fa-142065b7d647 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:40.452498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 573/1148] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:40.458936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8a5de-080e-428e-aad4-83f1e5446b0a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:40.459658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8a5de-080e-428e-aad4-83f1e5446b0a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:40.459843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8a5de-080e-428e-aad4-83f1e5446b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:40.538208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-daa8a5de-080e-428e-aad4-83f1e5446b0a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:40.539538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-daa8a5de-080e-428e-aad4-83f1e5446b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:40.649057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 576/1149] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:40.946975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127c41-b9b3-43fe-859b-ab09a901bb55 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:40.947259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127c41-b9b3-43fe-859b-ab09a901bb55 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:40.947533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127c41-b9b3-43fe-859b-ab09a901bb55 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:40.948880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 574/1150] 10.222.0.25 () {54 vars in 962 bytes} [Wed Nov 2 07:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:40.954206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd53635-a29e-4e11-82a2-6d1b32354abf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:40.954503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd53635-a29e-4e11-82a2-6d1b32354abf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:40.954775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd53635-a29e-4e11-82a2-6d1b32354abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:41.039883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2dd53635-a29e-4e11-82a2-6d1b32354abf None None] MFA Rules not processed for user `f9fd09151e284f639c93699b0d26db05`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:41.041086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2dd53635-a29e-4e11-82a2-6d1b32354abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:41.063068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:41.119051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 577/1151] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Nov 2 07:22:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:41.127231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29127c41-b9b3-43fe-859b-ab09a901bb55 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:41.128745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-29127c41-b9b3-43fe-859b-ab09a901bb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:41.247178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29127c41-b9b3-43fe-859b-ab09a901bb55 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:41.335213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be03ce-5daf-46b1-8760-a5ef5dc23e95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fd09151e284f639c93699b0d26db05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:41.335822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be03ce-5daf-46b1-8760-a5ef5dc23e95 service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:41.336095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be03ce-5daf-46b1-8760-a5ef5dc23e95 service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:41.336367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be03ce-5daf-46b1-8760-a5ef5dc23e95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:41.338346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7be03ce-5daf-46b1-8760-a5ef5dc23e95 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:41.406453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:41.471377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be03ce-5daf-46b1-8760-a5ef5dc23e95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:41.472824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be03ce-5daf-46b1-8760-a5ef5dc23e95 service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:41.495077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 575/1152] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:43.786850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af7f5b-789a-4801-9c62-14d1fc5534b4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:43.787844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af7f5b-789a-4801-9c62-14d1fc5534b4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:43.787844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af7f5b-789a-4801-9c62-14d1fc5534b4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:43.789771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 578/1153] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:22:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:43.795578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c948a-738a-49b5-88f1-1e28c6a5a862 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:43.796039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c948a-738a-49b5-88f1-1e28c6a5a862 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:43.796147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c948a-738a-49b5-88f1-1e28c6a5a862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:43.864792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-549c948a-738a-49b5-88f1-1e28c6a5a862 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:43.865986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-549c948a-738a-49b5-88f1-1e28c6a5a862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:43.966401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 576/1154] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:44.238981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af7f5b-789a-4801-9c62-14d1fc5534b4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:44.240540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c4af7f5b-789a-4801-9c62-14d1fc5534b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:44.367720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af7f5b-789a-4801-9c62-14d1fc5534b4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:44.416900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:44.464120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c92cc-8086-4810-8dc7-0f1ece3ee595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:44.464715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c92cc-8086-4810-8dc7-0f1ece3ee595 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:44.464984 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c92cc-8086-4810-8dc7-0f1ece3ee595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:44.465249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c92cc-8086-4810-8dc7-0f1ece3ee595 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:44.465882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c92cc-8086-4810-8dc7-0f1ece3ee595 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:44.467018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-396c92cc-8086-4810-8dc7-0f1ece3ee595 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:22:44.467018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:22:44.467018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:22:44.468136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c92cc-8086-4810-8dc7-0f1ece3ee595 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:44.487134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 579/1155] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Nov 2 07:22:44 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:22:46.651168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c5d4-99a9-41da-a3f8-bf5f5eb2839a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:46.651790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c5d4-99a9-41da-a3f8-bf5f5eb2839a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:46.651790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c5d4-99a9-41da-a3f8-bf5f5eb2839a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:46.653052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 577/1156] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:46.658624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ea08-f6ee-4b89-a028-f3972db163b4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:46.659094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ea08-f6ee-4b89-a028-f3972db163b4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:46.659196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ea08-f6ee-4b89-a028-f3972db163b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:46.732405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7cd1ea08-f6ee-4b89-a028-f3972db163b4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:46.733723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cd1ea08-f6ee-4b89-a028-f3972db163b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:46.835160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 580/1157] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:47.130255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7c5d4-99a9-41da-a3f8-bf5f5eb2839a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:47.131826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b1c7c5d4-99a9-41da-a3f8-bf5f5eb2839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:47.141961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:47.286335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7c5d4-99a9-41da-a3f8-bf5f5eb2839a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:47.373306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:47.407198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b74ed-1a5b-4ab5-9983-225e12cf6e00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:47.407801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b74ed-1a5b-4ab5-9983-225e12cf6e00 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:47.408070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b74ed-1a5b-4ab5-9983-225e12cf6e00 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:47.408335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b74ed-1a5b-4ab5-9983-225e12cf6e00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:47.409922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2a8b74ed-1a5b-4ab5-9983-225e12cf6e00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:47.511381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b74ed-1a5b-4ab5-9983-225e12cf6e00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:47.512970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b74ed-1a5b-4ab5-9983-225e12cf6e00 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:47.532055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 578/1158] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:49.891875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283cad-b92d-4ff4-9045-bfff28b1df1f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:49.892511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283cad-b92d-4ff4-9045-bfff28b1df1f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:49.892511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283cad-b92d-4ff4-9045-bfff28b1df1f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:49.893786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 581/1159] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:22:49.900473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed033a5-3aec-4b4c-8a1a-83b38a49a7f4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:49.900951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed033a5-3aec-4b4c-8a1a-83b38a49a7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:49.901063 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed033a5-3aec-4b4c-8a1a-83b38a49a7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:49.969889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6ed033a5-3aec-4b4c-8a1a-83b38a49a7f4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:49.971123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6ed033a5-3aec-4b4c-8a1a-83b38a49a7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:50.021177 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:50.075898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 579/1160] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:50.362821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283cad-b92d-4ff4-9045-bfff28b1df1f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:50.365282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-56283cad-b92d-4ff4-9045-bfff28b1df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:50.396943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:50.494481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283cad-b92d-4ff4-9045-bfff28b1df1f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:50.580217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03915a50-b4ba-448d-8e43-9c55164894b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:50.580825 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03915a50-b4ba-448d-8e43-9c55164894b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:50.581102 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03915a50-b4ba-448d-8e43-9c55164894b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:50.581367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03915a50-b4ba-448d-8e43-9c55164894b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:50.583010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-03915a50-b4ba-448d-8e43-9c55164894b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:50.664937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:50.701859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03915a50-b4ba-448d-8e43-9c55164894b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:50.703337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03915a50-b4ba-448d-8e43-9c55164894b7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:50.722117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 582/1161] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:52.926068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5e9d-e379-41ec-a070-a3e55585ea62 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:52.926614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5e9d-e379-41ec-a070-a3e55585ea62 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:52.926614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5e9d-e379-41ec-a070-a3e55585ea62 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:52.928002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 580/1162] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:52.933650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925f6f1-bab1-44b3-8649-02ce3c83732e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:52.934144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925f6f1-bab1-44b3-8649-02ce3c83732e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:52.934237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925f6f1-bab1-44b3-8649-02ce3c83732e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:53.002040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f925f6f1-bab1-44b3-8649-02ce3c83732e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:53.003289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f925f6f1-bab1-44b3-8649-02ce3c83732e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:53.105956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 583/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:53.393808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b5e9d-e379-41ec-a070-a3e55585ea62 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:53.395382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c40b5e9d-e379-41ec-a070-a3e55585ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:53.513132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b5e9d-e379-41ec-a070-a3e55585ea62 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:53.589459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:53.591900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0119-2b10-4fe5-abfc-369ab1b8b5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:53.592573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0119-2b10-4fe5-abfc-369ab1b8b5de service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:53.592915 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0119-2b10-4fe5-abfc-369ab1b8b5de service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:53.593646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0119-2b10-4fe5-abfc-369ab1b8b5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:53.595824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-408c0119-2b10-4fe5-abfc-369ab1b8b5de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:53.710958 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c0119-2b10-4fe5-abfc-369ab1b8b5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:53.712319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c0119-2b10-4fe5-abfc-369ab1b8b5de service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:53.735788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 581/1164] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:22:55.971868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f195b2-ab28-4b4b-be8c-125e898c7813 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:55.972367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f195b2-ab28-4b4b-be8c-125e898c7813 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:55.972367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f195b2-ab28-4b4b-be8c-125e898c7813 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:55.973770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 584/1165] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:55.980309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5af6-a856-48d6-b72a-771151fd4309 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:55.981101 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5af6-a856-48d6-b72a-771151fd4309 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:55.981243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5af6-a856-48d6-b72a-771151fd4309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:56.059434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-966e5af6-a856-48d6-b72a-771151fd4309 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:56.060651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-966e5af6-a856-48d6-b72a-771151fd4309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:56.116807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:56.163030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 582/1166] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:56.462606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f195b2-ab28-4b4b-be8c-125e898c7813 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:56.465449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33f195b2-ab28-4b4b-be8c-125e898c7813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:56.510408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:56.586401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f195b2-ab28-4b4b-be8c-125e898c7813 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:56.665895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492f46d-14f9-425a-88b5-8225f876ac9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:56.666550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492f46d-14f9-425a-88b5-8225f876ac9d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:56.666820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492f46d-14f9-425a-88b5-8225f876ac9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:56.667084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492f46d-14f9-425a-88b5-8225f876ac9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:56.668804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5492f46d-14f9-425a-88b5-8225f876ac9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:56.751480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:56.783344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492f46d-14f9-425a-88b5-8225f876ac9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:56.784824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492f46d-14f9-425a-88b5-8225f876ac9d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:56.805502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 585/1167] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:59.184285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17892700-fcf7-4c47-a695-643ea35477ec None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:59.184826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17892700-fcf7-4c47-a695-643ea35477ec None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:59.184826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17892700-fcf7-4c47-a695-643ea35477ec None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:59.186099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 583/1168] 10.222.0.25 () {54 vars in 927 bytes} [Wed Nov 2 07:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:59.195188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc4bdd-626e-440d-8cac-aee2c3394bdd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:59.195653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc4bdd-626e-440d-8cac-aee2c3394bdd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:59.195748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc4bdd-626e-440d-8cac-aee2c3394bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:59.265665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-adcc4bdd-626e-440d-8cac-aee2c3394bdd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:59.266909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-adcc4bdd-626e-440d-8cac-aee2c3394bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:59.366047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 586/1169] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:59.657151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17892700-fcf7-4c47-a695-643ea35477ec None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:59.658732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-17892700-fcf7-4c47-a695-643ea35477ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:59.765538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17892700-fcf7-4c47-a695-643ea35477ec None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:59.849100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a32cc-6736-4146-a395-0ae270ab614e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:59.849722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a32cc-6736-4146-a395-0ae270ab614e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:59.849994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a32cc-6736-4146-a395-0ae270ab614e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:59.850310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a32cc-6736-4146-a395-0ae270ab614e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:59.851950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-548a32cc-6736-4146-a395-0ae270ab614e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:59.861420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:59.971734 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a32cc-6736-4146-a395-0ae270ab614e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:59.973201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a32cc-6736-4146-a395-0ae270ab614e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:59.992591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 584/1170] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:09.364503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:09.989872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=50 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:13.662359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f2436-a637-4111-b7f5-46dae99af37e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:13.663005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f2436-a637-4111-b7f5-46dae99af37e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:13.663005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f2436-a637-4111-b7f5-46dae99af37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:13.747037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-988f2436-a637-4111-b7f5-46dae99af37e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:13.748177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-988f2436-a637-4111-b7f5-46dae99af37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:13.857633 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 587/1171] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:13.870702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3ac57-b728-4aae-8e0c-b08e7afda791 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:13.871259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3ac57-b728-4aae-8e0c-b08e7afda791 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:13.871792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3ac57-b728-4aae-8e0c-b08e7afda791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:13.958904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-14f3ac57-b728-4aae-8e0c-b08e7afda791 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:13.960105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-14f3ac57-b728-4aae-8e0c-b08e7afda791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:14.065642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 585/1172] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:14.070726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f2436-a637-4111-b7f5-46dae99af37e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:14.071951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-988f2436-a637-4111-b7f5-46dae99af37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:14.084455 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f2436-a637-4111-b7f5-46dae99af37e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:14.159033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:14.195793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:14.196410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:14.196670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:14.196934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:14.197572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:24:14.197868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:14.198918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:14.198918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:24:14.198918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:24:14.199801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:14.213103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 588/1173] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:24:14.223306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3ac57-b728-4aae-8e0c-b08e7afda791 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:14.233910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3ac57-b728-4aae-8e0c-b08e7afda791 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:14.299160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:14.337786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:14.338357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:14.338632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:14.338901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:14.339508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:14.341039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:14.341039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:24:14.341039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: Nov 02 07:24:14.342691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:14.417875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 586/1174] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:24:14 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:24:14.422413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:14.423969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:14.458929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f9a5b-7782-4ccb-9639-4bb03feedec3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:14.560897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:14.561918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:14.561918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:14.562169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:14.562741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:14.563763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:14.563763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:24:14.563763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:24:14.564957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:14.618619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 589/1175] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:24:14 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:24:14.624182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:14.625714 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:14.693732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637f674-e6fc-4308-98cc-79f9a2fee31d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:14.781712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:14.793616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:14.794259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:14.794535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:14.794804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:14.795444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:14.797245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:14.809565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 587/1176] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:24:14.814443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:14.815978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:14.828486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f27531-6a8b-4ac6-ac64-d6522f8cd14e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:14.854628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:14.928499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccefde-7dcd-45ae-8285-16e1f2ec8ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:14.929125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccefde-7dcd-45ae-8285-16e1f2ec8ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:14.929396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccefde-7dcd-45ae-8285-16e1f2ec8ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:14.929668 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccefde-7dcd-45ae-8285-16e1f2ec8ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:14.930363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccefde-7dcd-45ae-8285-16e1f2ec8ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:14.931998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccefde-7dcd-45ae-8285-16e1f2ec8ddf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:14.944988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 590/1177] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:24:14.949444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:14.950989 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:14.985712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40c2e-0049-4f20-8359-e9439210a7d0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:15.086891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d684d4-c448-43ee-aa90-9c71c64b1a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:15.087553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d684d4-c448-43ee-aa90-9c71c64b1a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:15.087841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d684d4-c448-43ee-aa90-9c71c64b1a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:15.088107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d684d4-c448-43ee-aa90-9c71c64b1a16 admin admin] PATH_INFO: `/v3/projects/a7a6e1e7e5714826b7f50a720acce7e3/users/fa5ca1ae8844479bb31042eb63b9e14c/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:15.112521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d684d4-c448-43ee-aa90-9c71c64b1a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a6e1e7e5714826b7f50a720acce7e3, user_id=fa5ca1ae8844479bb31042eb63b9e14c, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:15.115219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d684d4-c448-43ee-aa90-9c71c64b1a16 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:15.144352 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 588/1178] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:24:14 2022] PUT /identity/v3/projects/a7a6e1e7e5714826b7f50a720acce7e3/users/fa5ca1ae8844479bb31042eb63b9e14c/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:24:15.155796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c34d2-ab94-4ce6-b22f-3c56788e58a1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:15.156080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c34d2-ab94-4ce6-b22f-3c56788e58a1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:15.156348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c34d2-ab94-4ce6-b22f-3c56788e58a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:15.218545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8a8c34d2-ab94-4ce6-b22f-3c56788e58a1 None None] MFA Rules not processed for user `fa5ca1ae8844479bb31042eb63b9e14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:15.219708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a8c34d2-ab94-4ce6-b22f-3c56788e58a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:15.313638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 591/1179] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:24:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:15.327454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71e80-23be-41df-bad6-b95de597b981 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:15.327712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71e80-23be-41df-bad6-b95de597b981 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:15.328009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71e80-23be-41df-bad6-b95de597b981 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:15.329397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 589/1180] 10.222.0.25 () {54 vars in 963 bytes} [Wed Nov 2 07:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:15.335209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3cfdf-f292-41c0-93f7-130d6c1b9ce3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:15.335575 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3cfdf-f292-41c0-93f7-130d6c1b9ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:15.335952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3cfdf-f292-41c0-93f7-130d6c1b9ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:15.373047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:15.419053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c2b3cfdf-f292-41c0-93f7-130d6c1b9ce3 None None] MFA Rules not processed for user `d17ba458c5004646b5e21cabcb54fd20`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:15.420281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c2b3cfdf-f292-41c0-93f7-130d6c1b9ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:15.499086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 592/1181] 10.222.0.25 () {58 vars in 1060 bytes} [Wed Nov 2 07:24:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:15.507082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b71e80-23be-41df-bad6-b95de597b981 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:15.508621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-58b71e80-23be-41df-bad6-b95de597b981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:15.566444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:15.625093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b71e80-23be-41df-bad6-b95de597b981 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:15.706361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:15.706987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:15.707233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:15.707502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:15.709078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:15.815601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:15.816963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:15.820440 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:15.838871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 590/1182] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:15.897473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43940f9-1da5-4dd0-9b8b-093f444eada2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:15.897962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43940f9-1da5-4dd0-9b8b-093f444eada2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:15.898117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43940f9-1da5-4dd0-9b8b-093f444eada2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:15.900748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 593/1183] 10.222.0.25 () {52 vars in 830 bytes} [Wed Nov 2 07:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:15.904701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:15.905993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:16.022360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95af04-e4e5-4247-a259-6fd7f9dcec8e service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:16.125437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:16.126079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:16.126390 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:16.126654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] PATH_INFO: `/v3/users/fa5ca1ae8844479bb31042eb63b9e14c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:16.141365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5ca1ae8844479bb31042eb63b9e14c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:16.142432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:16.142432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:24:16.142432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: Nov 02 07:24:16.143634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:16.153575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:16.218832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5ca1ae8844479bb31042eb63b9e14c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:16.219270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5ca1ae8844479bb31042eb63b9e14c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:16.244060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5ca1ae8844479bb31042eb63b9e14c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:16.259396 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5ca1ae8844479bb31042eb63b9e14c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:16.259964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5ca1ae8844479bb31042eb63b9e14c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:16.276500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 591/1184] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:24:15 2022] DELETE /identity/v3/users/fa5ca1ae8844479bb31042eb63b9e14c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:24:16.281159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afaf4-faf7-4715-990c-e65d1db2fa4e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:16.281518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afaf4-faf7-4715-990c-e65d1db2fa4e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:16.281787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829afaf4-faf7-4715-990c-e65d1db2fa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:16.342552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-829afaf4-faf7-4715-990c-e65d1db2fa4e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:16.344010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-829afaf4-faf7-4715-990c-e65d1db2fa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:16.419111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:16.432086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 594/1185] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:16.443894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:16.445140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:16.551671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8d4c2-3a67-43ef-9197-286c94a6e2ad admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:16.611241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:16.631901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3a02e-95d8-4a7a-81ab-9ad3cd9a7f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:16.632518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3a02e-95d8-4a7a-81ab-9ad3cd9a7f72 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:16.632784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3a02e-95d8-4a7a-81ab-9ad3cd9a7f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:16.633066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3a02e-95d8-4a7a-81ab-9ad3cd9a7f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:16.634702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-17a3a02e-95d8-4a7a-81ab-9ad3cd9a7f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:16.756283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3a02e-95d8-4a7a-81ab-9ad3cd9a7f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:16.757617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3a02e-95d8-4a7a-81ab-9ad3cd9a7f72 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:16.781939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 592/1186] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:17.150287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829afaf4-faf7-4715-990c-e65d1db2fa4e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:17.151663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-829afaf4-faf7-4715-990c-e65d1db2fa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:17.163298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829afaf4-faf7-4715-990c-e65d1db2fa4e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:17.279469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:17.280020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:17.280299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:17.280568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] PATH_INFO: `/v3/projects/a7a6e1e7e5714826b7f50a720acce7e3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:17.292109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a6e1e7e5714826b7f50a720acce7e3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:17.293168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:17.293168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:24:17.293168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:24:17.294408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:17.368555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a6e1e7e5714826b7f50a720acce7e3', 'request_id': 'req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:17.382055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a6e1e7e5714826b7f50a720acce7e3', 'request_id': 'req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:17.393057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a6e1e7e5714826b7f50a720acce7e3', 'request_id': 'req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:17.425520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] The token cache is being invalidate because project a7a6e1e7e5714826b7f50a720acce7e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:24:17.425813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:17.426369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-739d7cb2-c564-4743-b4c2-6ddf56d1dee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:17.427676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 595/1187] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:24:17 2022] DELETE /identity/v3/projects/a7a6e1e7e5714826b7f50a720acce7e3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:24:26.778734 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=62 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:27.423425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:44.164134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b428d-ad5f-44d7-b6e9-d750d783b408 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:44.164725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b428d-ad5f-44d7-b6e9-d750d783b408 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:44.164725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b428d-ad5f-44d7-b6e9-d750d783b408 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:44.166200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 593/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:44.174343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5e1bb-63f7-47d5-aca6-d1d789b4b8fc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:44.175093 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5e1bb-63f7-47d5-aca6-d1d789b4b8fc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:44.175245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5e1bb-63f7-47d5-aca6-d1d789b4b8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:44.267887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7ad5e1bb-63f7-47d5-aca6-d1d789b4b8fc None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:44.269120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7ad5e1bb-63f7-47d5-aca6-d1d789b4b8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:44.376418 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 596/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:44.539835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b428d-ad5f-44d7-b6e9-d750d783b408 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:44.542302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-333b428d-ad5f-44d7-b6e9-d750d783b408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:44.669550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b428d-ad5f-44d7-b6e9-d750d783b408 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:44.679846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38ab16-1eb8-46f0-9386-9ea8908fc4fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:44.680686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ab16-1eb8-46f0-9386-9ea8908fc4fe service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:44.680997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ab16-1eb8-46f0-9386-9ea8908fc4fe service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:44.681518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ab16-1eb8-46f0-9386-9ea8908fc4fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:44.694451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38ab16-1eb8-46f0-9386-9ea8908fc4fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:44.696059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38ab16-1eb8-46f0-9386-9ea8908fc4fe service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:44.705689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 594/1190] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:25:45.599119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c1870-ad2e-4a16-b04e-8ee9b94565aa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:45.599711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c1870-ad2e-4a16-b04e-8ee9b94565aa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:45.599711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c1870-ad2e-4a16-b04e-8ee9b94565aa None None] PATH_INFO: `/v3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:45.600961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 597/1191] 10.222.0.46 () {54 vars in 925 bytes} [Wed Nov 2 07:25:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:25:45.607453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5322-d189-4913-be48-2c9f90b9d104 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:45.607926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5322-d189-4913-be48-2c9f90b9d104 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:45.608033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5322-d189-4913-be48-2c9f90b9d104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:45.670616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-915b5322-d189-4913-be48-2c9f90b9d104 None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:45.671804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-915b5322-d189-4913-be48-2c9f90b9d104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:45.744148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:45.749639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 595/1192] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Nov 2 07:25:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:45.768743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cd825-356e-451d-a30d-57e67b49ccac None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:45.769075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cd825-356e-451d-a30d-57e67b49ccac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:45.769503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cd825-356e-451d-a30d-57e67b49ccac None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:45.772194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 598/1193] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:25:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:45.776998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd7977-13d3-4fea-8014-cc0f4065c8d7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:45.777278 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd7977-13d3-4fea-8014-cc0f4065c8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:45.777552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd7977-13d3-4fea-8014-cc0f4065c8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:45.797856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-efcd7977-13d3-4fea-8014-cc0f4065c8d7 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:45.799103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-efcd7977-13d3-4fea-8014-cc0f4065c8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:45.808843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 596/1194] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:25:45.958708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067cd825-356e-451d-a30d-57e67b49ccac None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:45.977216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067cd825-356e-451d-a30d-57e67b49ccac None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:45.983159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50c87a-cdbc-47be-8e3d-dc5c1359d088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:45.983779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50c87a-cdbc-47be-8e3d-dc5c1359d088 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:45.984053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50c87a-cdbc-47be-8e3d-dc5c1359d088 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:45.984318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50c87a-cdbc-47be-8e3d-dc5c1359d088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:45.992714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50c87a-cdbc-47be-8e3d-dc5c1359d088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:45.994322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50c87a-cdbc-47be-8e3d-dc5c1359d088 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:46.003583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 599/1195] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:25:54.408123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fd89d-8483-44f9-9260-95582c5bb613 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:54.408613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fd89d-8483-44f9-9260-95582c5bb613 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:54.408716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fd89d-8483-44f9-9260-95582c5bb613 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:54.410556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 597/1196] 10.222.0.25 () {54 vars in 961 bytes} [Wed Nov 2 07:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:54.416583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c402f0b-b623-4b72-ada5-94bbd0175d61 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:54.417041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c402f0b-b623-4b72-ada5-94bbd0175d61 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:54.417135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c402f0b-b623-4b72-ada5-94bbd0175d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:54.438485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0c402f0b-b623-4b72-ada5-94bbd0175d61 None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:54.439690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c402f0b-b623-4b72-ada5-94bbd0175d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:54.449541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 600/1197] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Nov 2 07:25:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:54.457630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fd89d-8483-44f9-9260-95582c5bb613 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:54.471802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fd89d-8483-44f9-9260-95582c5bb613 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:54.477197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20422301-edd5-475e-983b-6731ac144b52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:54.477933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422301-edd5-475e-983b-6731ac144b52 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:54.478273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422301-edd5-475e-983b-6731ac144b52 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:54.478546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422301-edd5-475e-983b-6731ac144b52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:54.486763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20422301-edd5-475e-983b-6731ac144b52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:54.488322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20422301-edd5-475e-983b-6731ac144b52 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:54.498395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 598/1198] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:25:57.489500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef888c-d4f7-4d28-8421-349136f04bc7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:57.490225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef888c-d4f7-4d28-8421-349136f04bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:57.490225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef888c-d4f7-4d28-8421-349136f04bc7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:57.491531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 601/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:57.497360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05e0e-4a83-442b-a5d6-684b36ac5b02 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:57.497824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05e0e-4a83-442b-a5d6-684b36ac5b02 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:57.497919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05e0e-4a83-442b-a5d6-684b36ac5b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:57.521186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fbb05e0e-4a83-442b-a5d6-684b36ac5b02 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:57.522382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fbb05e0e-4a83-442b-a5d6-684b36ac5b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:57.530648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 599/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:57.684656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef888c-d4f7-4d28-8421-349136f04bc7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:57.699582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef888c-d4f7-4d28-8421-349136f04bc7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:57.705755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842690cf-38c2-4e51-93ad-db1094f0e1c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:57.706386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842690cf-38c2-4e51-93ad-db1094f0e1c6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:57.706668 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842690cf-38c2-4e51-93ad-db1094f0e1c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:57.706943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842690cf-38c2-4e51-93ad-db1094f0e1c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:57.715530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842690cf-38c2-4e51-93ad-db1094f0e1c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:57.717471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842690cf-38c2-4e51-93ad-db1094f0e1c6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:57.727064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 602/1201] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:25:58.778546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a036c-a2d6-48fb-8421-1e73a1d390e9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:58.779502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a036c-a2d6-48fb-8421-1e73a1d390e9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:58.779502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a036c-a2d6-48fb-8421-1e73a1d390e9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:58.781540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 600/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:25:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:58.787496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809dcc9-a059-4c1e-a65f-bd1d72f13e11 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:58.787963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809dcc9-a059-4c1e-a65f-bd1d72f13e11 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:58.788063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809dcc9-a059-4c1e-a65f-bd1d72f13e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:58.808887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c809dcc9-a059-4c1e-a65f-bd1d72f13e11 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:58.810130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c809dcc9-a059-4c1e-a65f-bd1d72f13e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:58.819563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 603/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:58.972023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a036c-a2d6-48fb-8421-1e73a1d390e9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:58.988585 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a036c-a2d6-48fb-8421-1e73a1d390e9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:58.995074 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deaa30-9576-4531-bce7-486ee099936d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:58.995667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deaa30-9576-4531-bce7-486ee099936d service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:58.995939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deaa30-9576-4531-bce7-486ee099936d service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:58.996205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deaa30-9576-4531-bce7-486ee099936d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:59.005269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deaa30-9576-4531-bce7-486ee099936d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:59.006946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deaa30-9576-4531-bce7-486ee099936d service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:59.016981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 601/1204] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:02.988996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed709e0d-6eeb-48af-83ae-bc23c5155ae3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:02.989948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed709e0d-6eeb-48af-83ae-bc23c5155ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:02.990064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed709e0d-6eeb-48af-83ae-bc23c5155ae3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:02.991447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 604/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:02.997819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bccba-acba-42bc-bb56-7dd409f1cca2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:02.998383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bccba-acba-42bc-bb56-7dd409f1cca2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:02.998495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bccba-acba-42bc-bb56-7dd409f1cca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:03.027909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c69bccba-acba-42bc-bb56-7dd409f1cca2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:03.029104 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c69bccba-acba-42bc-bb56-7dd409f1cca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:03.038629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 602/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:03.323246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed709e0d-6eeb-48af-83ae-bc23c5155ae3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:03.326137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed709e0d-6eeb-48af-83ae-bc23c5155ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:03.374242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:26:03.423170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed709e0d-6eeb-48af-83ae-bc23c5155ae3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:03.427223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8744ba6-5ce5-4614-ae3f-bf8210fbf635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:03.427864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8744ba6-5ce5-4614-ae3f-bf8210fbf635 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:03.428177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8744ba6-5ce5-4614-ae3f-bf8210fbf635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:03.428412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8744ba6-5ce5-4614-ae3f-bf8210fbf635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:03.439666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8744ba6-5ce5-4614-ae3f-bf8210fbf635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:03.441149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8744ba6-5ce5-4614-ae3f-bf8210fbf635 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:03.451007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 605/1207] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:05.571178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc65c18-e65c-4215-8b17-6293c3f254d9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:05.572079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc65c18-e65c-4215-8b17-6293c3f254d9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:05.572079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc65c18-e65c-4215-8b17-6293c3f254d9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:05.574014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 603/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:05.580386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f94b-7a94-4cb6-b3ba-b7806b4c244e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:05.581113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f94b-7a94-4cb6-b3ba-b7806b4c244e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:05.581229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f94b-7a94-4cb6-b3ba-b7806b4c244e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:05.606220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ecb6f94b-7a94-4cb6-b3ba-b7806b4c244e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:05.607412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ecb6f94b-7a94-4cb6-b3ba-b7806b4c244e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:05.618008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 606/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:05.900995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc65c18-e65c-4215-8b17-6293c3f254d9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:05.903240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ddc65c18-e65c-4215-8b17-6293c3f254d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:05.918182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc65c18-e65c-4215-8b17-6293c3f254d9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:05.922071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbf21f-84bc-4597-8c87-5a3f68aa33a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:05.922701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbf21f-84bc-4597-8c87-5a3f68aa33a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:05.922969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbf21f-84bc-4597-8c87-5a3f68aa33a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:05.923238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbf21f-84bc-4597-8c87-5a3f68aa33a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:05.932348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbf21f-84bc-4597-8c87-5a3f68aa33a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:05.933764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbf21f-84bc-4597-8c87-5a3f68aa33a7 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:05.942956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 604/1210] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:09.625921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6d90-e9ae-48cf-b200-0c43e6e130f1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:09.626831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6d90-e9ae-48cf-b200-0c43e6e130f1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:09.626831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6d90-e9ae-48cf-b200-0c43e6e130f1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:09.628660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 607/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:09.634994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd910a-f7ee-45b9-99d9-aa1bff3b97c0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:09.635557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd910a-f7ee-45b9-99d9-aa1bff3b97c0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:09.635557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd910a-f7ee-45b9-99d9-aa1bff3b97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:09.658438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-08fd910a-f7ee-45b9-99d9-aa1bff3b97c0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:09.659632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-08fd910a-f7ee-45b9-99d9-aa1bff3b97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:09.668120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 605/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:09.972338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c6d90-e9ae-48cf-b200-0c43e6e130f1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:09.975373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4a6c6d90-e9ae-48cf-b200-0c43e6e130f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:10.056716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c6d90-e9ae-48cf-b200-0c43e6e130f1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:10.061011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd2e6-2fde-43c6-bbdc-3526717ae0ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:10.061123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd2e6-2fde-43c6-bbdc-3526717ae0ea service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:10.061370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd2e6-2fde-43c6-bbdc-3526717ae0ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:10.061647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd2e6-2fde-43c6-bbdc-3526717ae0ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:10.071280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abd2e6-2fde-43c6-bbdc-3526717ae0ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:10.072716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abd2e6-2fde-43c6-bbdc-3526717ae0ea service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:10.082205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 608/1213] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:12.250043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba67716-5455-48d6-ae05-6deb62bb3962 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:12.250956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba67716-5455-48d6-ae05-6deb62bb3962 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:12.250956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba67716-5455-48d6-ae05-6deb62bb3962 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:12.253084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 606/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:26:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:12.258808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a1f13-9cc5-4e8a-b822-a22bcd8464d6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:12.259276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a1f13-9cc5-4e8a-b822-a22bcd8464d6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:12.259376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a1f13-9cc5-4e8a-b822-a22bcd8464d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:12.281411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6b6a1f13-9cc5-4e8a-b822-a22bcd8464d6 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:12.282642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6b6a1f13-9cc5-4e8a-b822-a22bcd8464d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:12.291377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 609/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:12.612480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba67716-5455-48d6-ae05-6deb62bb3962 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:12.614736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6ba67716-5455-48d6-ae05-6deb62bb3962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:12.629936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba67716-5455-48d6-ae05-6deb62bb3962 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:12.634488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f68c8-816c-4667-8a99-484809b07222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:12.635083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f68c8-816c-4667-8a99-484809b07222 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:12.635357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f68c8-816c-4667-8a99-484809b07222 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:12.635628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f68c8-816c-4667-8a99-484809b07222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:12.647174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f68c8-816c-4667-8a99-484809b07222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:12.648660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f68c8-816c-4667-8a99-484809b07222 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:12.658289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 607/1216] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:22.276112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=63 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:26:22.654300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=37 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:38.669818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d8077-6da2-4270-b44e-54b2fd0d8b60 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.670520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d8077-6da2-4270-b44e-54b2fd0d8b60 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.670520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d8077-6da2-4270-b44e-54b2fd0d8b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.682571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad4e4-6c5e-4580-9e6a-c29cb1cc45d5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.683216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad4e4-6c5e-4580-9e6a-c29cb1cc45d5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.683325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad4e4-6c5e-4580-9e6a-c29cb1cc45d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.695650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-740d8077-6da2-4270-b44e-54b2fd0d8b60 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:38.696871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-740d8077-6da2-4270-b44e-54b2fd0d8b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:38.712620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 610/1217] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:38.719341 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-11aad4e4-6c5e-4580-9e6a-c29cb1cc45d5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:38.721706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-11aad4e4-6c5e-4580-9e6a-c29cb1cc45d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:38.730884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ac230-84bf-40a8-98c2-c8687ed7b3f3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.731520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ac230-84bf-40a8-98c2-c8687ed7b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.731757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ac230-84bf-40a8-98c2-c8687ed7b3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.737120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 608/1218] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:38.738663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737dc7a-025b-47e6-8a72-6bceb6d23edf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.738946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737dc7a-025b-47e6-8a72-6bceb6d23edf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.739215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737dc7a-025b-47e6-8a72-6bceb6d23edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.764256 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6737dc7a-025b-47e6-8a72-6bceb6d23edf None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:38.765444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6737dc7a-025b-47e6-8a72-6bceb6d23edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:38.773794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4c5ac230-84bf-40a8-98c2-c8687ed7b3f3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:38.777786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4c5ac230-84bf-40a8-98c2-c8687ed7b3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:38.784809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 609/1219] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:38.786270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8b47e-4c70-4d59-934f-65741a1cc9e6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.786551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8b47e-4c70-4d59-934f-65741a1cc9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.786820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8b47e-4c70-4d59-934f-65741a1cc9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.801556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 611/1220] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:38.802559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ac230-84bf-40a8-98c2-c8687ed7b3f3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:38.816204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-efb8b47e-4c70-4d59-934f-65741a1cc9e6 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:38.817420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-efb8b47e-4c70-4d59-934f-65741a1cc9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:38.818901 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ac230-84bf-40a8-98c2-c8687ed7b3f3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:38.837856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 610/1221] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:38.838599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:38.839225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50006ede-86f0-46be-943e-74d1d9c64fce None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.839344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.839531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.839646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50006ede-86f0-46be-943e-74d1d9c64fce None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.839811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50006ede-86f0-46be-943e-74d1d9c64fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.839917 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.840414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:38.840714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:38.842438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:38.870214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 612/1222] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:38.871276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:38.872210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-50006ede-86f0-46be-943e-74d1d9c64fce None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:38.873721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-50006ede-86f0-46be-943e-74d1d9c64fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:38.888603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289fbab-6b72-4b30-92c9-f0238e8c5147 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:38.904025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 611/1223] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:38.904372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:38.905044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.905321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.905594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.906306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:38.906612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:38.907540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498437f7-521a-4cf3-a46a-30beb41f5913 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.908043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498437f7-521a-4cf3-a46a-30beb41f5913 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.908629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498437f7-521a-4cf3-a46a-30beb41f5913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.911669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:38.926631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 613/1224] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:38.927682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:38.944245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76095d69-6334-4dbc-a916-2ebb1f6634b1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:38.953005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-498437f7-521a-4cf3-a46a-30beb41f5913 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:38.955339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-498437f7-521a-4cf3-a46a-30beb41f5913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:38.956722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:38.957339 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.957611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.957885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:38.958544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:38.959558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:38.959558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:27:38.959558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:27:38.960732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:38.977801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 612/1225] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:38.979045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591f6d7-c67a-4a80-b6a0-7894c8b737e8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:38.979324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591f6d7-c67a-4a80-b6a0-7894c8b737e8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:38.979592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591f6d7-c67a-4a80-b6a0-7894c8b737e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.008223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6591f6d7-c67a-4a80-b6a0-7894c8b737e8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.009526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6591f6d7-c67a-4a80-b6a0-7894c8b737e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.025758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 613/1226] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.027397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591f6d7-c67a-4a80-b6a0-7894c8b737e8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.040362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591f6d7-c67a-4a80-b6a0-7894c8b737e8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.043767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 614/1227] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:38 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.044806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.132976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1a3a5-2b9d-45eb-b2d2-96fdd0bfb931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.133654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1a3a5-2b9d-45eb-b2d2-96fdd0bfb931 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.133949 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1a3a5-2b9d-45eb-b2d2-96fdd0bfb931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.134243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1a3a5-2b9d-45eb-b2d2-96fdd0bfb931 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.135029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1a3a5-2b9d-45eb-b2d2-96fdd0bfb931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:39.135855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1a3a5-2b9d-45eb-b2d2-96fdd0bfb931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.136884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-a3a1a3a5-2b9d-45eb-b2d2-96fdd0bfb931 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:39.136884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:27:39.136884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: Nov 02 07:27:39.137765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1a3a5-2b9d-45eb-b2d2-96fdd0bfb931 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.152510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 614/1228] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:39.153713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480432-1f11-40a6-88c3-0f8ae31e6733 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.153969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480432-1f11-40a6-88c3-0f8ae31e6733 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.154277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3480432-1f11-40a6-88c3-0f8ae31e6733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.174953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f3480432-1f11-40a6-88c3-0f8ae31e6733 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.176057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f3480432-1f11-40a6-88c3-0f8ae31e6733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.181839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f8395-40a0-4179-a8ce-62ed7f332b5b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.183732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 615/1229] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.184895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b4cc-6067-4db5-a906-fca4c4d376e5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.185171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b4cc-6067-4db5-a906-fca4c4d376e5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.185445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b4cc-6067-4db5-a906-fca4c4d376e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.191640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86c9fc-3fc3-4e60-9630-3b014fca1d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.192263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86c9fc-3fc3-4e60-9630-3b014fca1d39 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.192596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86c9fc-3fc3-4e60-9630-3b014fca1d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.192875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86c9fc-3fc3-4e60-9630-3b014fca1d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.193519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86c9fc-3fc3-4e60-9630-3b014fca1d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.196749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86c9fc-3fc3-4e60-9630-3b014fca1d39 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.205775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d2d7b4cc-6067-4db5-a906-fca4c4d376e5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.207000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d2d7b4cc-6067-4db5-a906-fca4c4d376e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.215228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 616/1230] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.216118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7b4cc-6067-4db5-a906-fca4c4d376e5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.225777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 615/1231] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.227122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7b4cc-6067-4db5-a906-fca4c4d376e5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.227364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e7e81-2a35-4346-a9a3-2238dcfdfb94 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.227649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e7e81-2a35-4346-a9a3-2238dcfdfb94 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.228029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e7e81-2a35-4346-a9a3-2238dcfdfb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.248884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-055e7e81-2a35-4346-a9a3-2238dcfdfb94 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.250271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-055e7e81-2a35-4346-a9a3-2238dcfdfb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.286932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5263c4-01d8-4dac-9239-619e128fcf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.287555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5263c4-01d8-4dac-9239-619e128fcf1b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.287834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5263c4-01d8-4dac-9239-619e128fcf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.288425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5263c4-01d8-4dac-9239-619e128fcf1b admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.288728 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5263c4-01d8-4dac-9239-619e128fcf1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.289745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-fe5263c4-01d8-4dac-9239-619e128fcf1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:39.289745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:27:39.289745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: Nov 02 07:27:39.290993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5263c4-01d8-4dac-9239-619e128fcf1b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.310071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 616/1232] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.311495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246344e-0e93-496f-9d39-0448911a8163 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.311773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246344e-0e93-496f-9d39-0448911a8163 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.312040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246344e-0e93-496f-9d39-0448911a8163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.316443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 617/1233] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.316443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8d7aa-dd3d-4924-a033-6b29fe4be39d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.316443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8d7aa-dd3d-4924-a033-6b29fe4be39d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.316443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8d7aa-dd3d-4924-a033-6b29fe4be39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.332663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c246344e-0e93-496f-9d39-0448911a8163 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.333874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c246344e-0e93-496f-9d39-0448911a8163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.336858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-11e8d7aa-dd3d-4924-a033-6b29fe4be39d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.337998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-11e8d7aa-dd3d-4924-a033-6b29fe4be39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.341781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 617/1234] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.342651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246344e-0e93-496f-9d39-0448911a8163 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.346701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 618/1235] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.347305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8d7aa-dd3d-4924-a033-6b29fe4be39d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.357810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8d7aa-dd3d-4924-a033-6b29fe4be39d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.358136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246344e-0e93-496f-9d39-0448911a8163 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.362613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.363228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.363496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.363750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7eed09-35ac-4087-b8b5-e620250236f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.363875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.364361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eed09-35ac-4087-b8b5-e620250236f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.364466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:39.364656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eed09-35ac-4087-b8b5-e620250236f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.364760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.364900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eed09-35ac-4087-b8b5-e620250236f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.365532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7eed09-35ac-4087-b8b5-e620250236f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.366320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.367489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7eed09-35ac-4087-b8b5-e620250236f0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.373710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:39.378677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 619/1236] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:39.379504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.391724 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6d1a-277f-4e85-b964-edadeb6ad76e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.396488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.397124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.397416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.397675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.401283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 618/1237] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.401440 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:39.401440 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.401603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7eed09-35ac-4087-b8b5-e620250236f0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.401684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.408406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:39.421064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 620/1238] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:39.421064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.425144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8b6d9-4c3d-4f1c-ac26-f89badeec656 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.461020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7eed09-35ac-4087-b8b5-e620250236f0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.465690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d1c3a-4d1f-46ef-b713-b61be83cb939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.466352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1c3a-4d1f-46ef-b713-b61be83cb939 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.466617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1c3a-4d1f-46ef-b713-b61be83cb939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.466891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d1c3a-4d1f-46ef-b713-b61be83cb939 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.467524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d1c3a-4d1f-46ef-b713-b61be83cb939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.469888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d1c3a-4d1f-46ef-b713-b61be83cb939 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.484442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6f9df-3f16-47fe-82c7-a33861d2bc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.485058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6f9df-3f16-47fe-82c7-a33861d2bc53 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.485315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6f9df-3f16-47fe-82c7-a33861d2bc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.485609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6f9df-3f16-47fe-82c7-a33861d2bc53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.486369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6f9df-3f16-47fe-82c7-a33861d2bc53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.487974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6f9df-3f16-47fe-82c7-a33861d2bc53 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.492108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 619/1239] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.493506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cebfa4-33ef-455b-9cc1-847d832e3b06 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.493822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cebfa4-33ef-455b-9cc1-847d832e3b06 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.494142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cebfa4-33ef-455b-9cc1-847d832e3b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.500418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 621/1240] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:39.501692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4425e83-ff9a-4d19-8ddc-b3642b0fecbd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.502186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4425e83-ff9a-4d19-8ddc-b3642b0fecbd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.502477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4425e83-ff9a-4d19-8ddc-b3642b0fecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.513552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-14cebfa4-33ef-455b-9cc1-847d832e3b06 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.514796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-14cebfa4-33ef-455b-9cc1-847d832e3b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.521397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f4425e83-ff9a-4d19-8ddc-b3642b0fecbd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.522575 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f4425e83-ff9a-4d19-8ddc-b3642b0fecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.522933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 620/1241] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.526626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77114e0-7a23-4945-a48e-58db40492aef None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.526626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77114e0-7a23-4945-a48e-58db40492aef None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.526626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77114e0-7a23-4945-a48e-58db40492aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.531299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 622/1242] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.532574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4425e83-ff9a-4d19-8ddc-b3642b0fecbd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.542669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4425e83-ff9a-4d19-8ddc-b3642b0fecbd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.544284 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e77114e0-7a23-4945-a48e-58db40492aef None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:39.545608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e77114e0-7a23-4945-a48e-58db40492aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:39.547325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8443a8f-1407-4cf3-af18-05024d8d2c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.548218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8443a8f-1407-4cf3-af18-05024d8d2c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.548345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8443a8f-1407-4cf3-af18-05024d8d2c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.548736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8443a8f-1407-4cf3-af18-05024d8d2c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.549381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8443a8f-1407-4cf3-af18-05024d8d2c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.551310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8443a8f-1407-4cf3-af18-05024d8d2c11 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.555165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 621/1243] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:39.556273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77114e0-7a23-4945-a48e-58db40492aef None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.567271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77114e0-7a23-4945-a48e-58db40492aef None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.574205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1152-658c-4bab-ac03-8d609fcc959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.575119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1152-658c-4bab-ac03-8d609fcc959d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.575405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1152-658c-4bab-ac03-8d609fcc959d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.575674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1152-658c-4bab-ac03-8d609fcc959d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.576306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb1152-658c-4bab-ac03-8d609fcc959d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.578259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb1152-658c-4bab-ac03-8d609fcc959d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.584130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 623/1244] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.584130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8443a8f-1407-4cf3-af18-05024d8d2c11 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.600610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 622/1245] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/users => generated 286 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.601465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1152-658c-4bab-ac03-8d609fcc959d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.610920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1152-658c-4bab-ac03-8d609fcc959d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.648625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8443a8f-1407-4cf3-af18-05024d8d2c11 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.653266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6e95-8586-435e-9e4b-d476f8fc148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.653878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6e95-8586-435e-9e4b-d476f8fc148c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.654216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6e95-8586-435e-9e4b-d476f8fc148c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.654466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6e95-8586-435e-9e4b-d476f8fc148c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.655220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f6e95-8586-435e-9e4b-d476f8fc148c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.657150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f6e95-8586-435e-9e4b-d476f8fc148c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.666626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9345f46-463a-4533-a3d5-00d01a256a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.667232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345f46-463a-4533-a3d5-00d01a256a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.667510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345f46-463a-4533-a3d5-00d01a256a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.667778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345f46-463a-4533-a3d5-00d01a256a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.668416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9345f46-463a-4533-a3d5-00d01a256a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.670012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9345f46-463a-4533-a3d5-00d01a256a47 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.683729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 623/1246] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:39.683729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9345f46-463a-4533-a3d5-00d01a256a47 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.684077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 624/1247] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.684992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6e95-8586-435e-9e4b-d476f8fc148c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.692454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9345f46-463a-4533-a3d5-00d01a256a47 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.732034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:39.736979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:39.749502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee78b7-2a6f-4c34-815d-3c039109ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.749690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6e95-8586-435e-9e4b-d476f8fc148c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.750142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee78b7-2a6f-4c34-815d-3c039109ff7a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.750422 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee78b7-2a6f-4c34-815d-3c039109ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.750690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee78b7-2a6f-4c34-815d-3c039109ff7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.751383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee78b7-2a6f-4c34-815d-3c039109ff7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.752986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee78b7-2a6f-4c34-815d-3c039109ff7a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.754862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.755552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.755916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.756244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.756979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:39.757336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.759017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.762945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 624/1248] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:39.763743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee78b7-2a6f-4c34-815d-3c039109ff7a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.772032 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 625/1249] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:39.772787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.773366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee78b7-2a6f-4c34-815d-3c039109ff7a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.777971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.778861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.778861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.779142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.779790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:39.780087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.781670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.783307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c079f4-b89e-4bd3-aa42-674244ce5860 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.787834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.788420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.788706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.788978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.789611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:39.789940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.791545 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.794067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 625/1250] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:39.794862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.810431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 626/1251] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:39.810431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.810745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8743f3e-9a6d-4c50-9370-13b2eb895178 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.811155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7340fca-55eb-47b3-aa3e-063054c3e749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.811747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7340fca-55eb-47b3-aa3e-063054c3e749 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.812020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7340fca-55eb-47b3-aa3e-063054c3e749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.812287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7340fca-55eb-47b3-aa3e-063054c3e749 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.812909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7340fca-55eb-47b3-aa3e-063054c3e749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.813934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7e502-3484-4244-b964-9d09de2ba60d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.814953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7340fca-55eb-47b3-aa3e-063054c3e749 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.818648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661854f-3255-42be-ad7b-a45bd49cf178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.819281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661854f-3255-42be-ad7b-a45bd49cf178 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.819578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661854f-3255-42be-ad7b-a45bd49cf178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.819846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661854f-3255-42be-ad7b-a45bd49cf178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.820480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661854f-3255-42be-ad7b-a45bd49cf178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.822042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661854f-3255-42be-ad7b-a45bd49cf178 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.832765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 627/1252] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:39.833651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661854f-3255-42be-ad7b-a45bd49cf178 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.842068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 626/1253] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.842068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7340fca-55eb-47b3-aa3e-063054c3e749 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.846625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661854f-3255-42be-ad7b-a45bd49cf178 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.850003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7340fca-55eb-47b3-aa3e-063054c3e749 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.851400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1724f6-3049-408f-968c-259c483487a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.852019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1724f6-3049-408f-968c-259c483487a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.852291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1724f6-3049-408f-968c-259c483487a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.852558 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1724f6-3049-408f-968c-259c483487a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.853195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1724f6-3049-408f-968c-259c483487a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.854806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1724f6-3049-408f-968c-259c483487a0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.854936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7329142-6e42-4eb4-85f2-8d7f74f8cb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.855418 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7329142-6e42-4eb4-85f2-8d7f74f8cb51 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.855694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7329142-6e42-4eb4-85f2-8d7f74f8cb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.856059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7329142-6e42-4eb4-85f2-8d7f74f8cb51 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.856572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7329142-6e42-4eb4-85f2-8d7f74f8cb51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.858490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7329142-6e42-4eb4-85f2-8d7f74f8cb51 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.865077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 628/1254] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:39.865761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1724f6-3049-408f-968c-259c483487a0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.875934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1724f6-3049-408f-968c-259c483487a0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.881945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d060491-07c9-4908-a5d3-3eb9d48c1787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.882643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d060491-07c9-4908-a5d3-3eb9d48c1787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.882927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d060491-07c9-4908-a5d3-3eb9d48c1787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.883302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 627/1255] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.883444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d060491-07c9-4908-a5d3-3eb9d48c1787 admin admin] PATH_INFO: `/v3/projects/02351bab6e5041a7969b50e93fa7d010/users/ec0e6cbf12de4edb9d3ab4ab0d2448b3/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.884158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7329142-6e42-4eb4-85f2-8d7f74f8cb51 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.895422 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7329142-6e42-4eb4-85f2-8d7f74f8cb51 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:39.899323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d060491-07c9-4908-a5d3-3eb9d48c1787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02351bab6e5041a7969b50e93fa7d010, user_id=ec0e6cbf12de4edb9d3ab4ab0d2448b3, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.901217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae2ad-547e-4955-823e-1529ea46a2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:39.901890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae2ad-547e-4955-823e-1529ea46a2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:39.902041 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d060491-07c9-4908-a5d3-3eb9d48c1787 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.902178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae2ad-547e-4955-823e-1529ea46a2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:39.902457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae2ad-547e-4955-823e-1529ea46a2a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:39.903088 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dae2ad-547e-4955-823e-1529ea46a2a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:39.904948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dae2ad-547e-4955-823e-1529ea46a2a1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:39.920195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 629/1256] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:39 2022] PUT /identity/v3/projects/02351bab6e5041a7969b50e93fa7d010/users/ec0e6cbf12de4edb9d3ab4ab0d2448b3/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:39.921468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d060491-07c9-4908-a5d3-3eb9d48c1787 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:39.948533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 628/1257] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:39.948533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae2ad-547e-4955-823e-1529ea46a2a1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.013816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d060491-07c9-4908-a5d3-3eb9d48c1787 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.035490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae2ad-547e-4955-823e-1529ea46a2a1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.040931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bbf26-ef03-46a9-b7d0-b1d45ea79b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.041538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bbf26-ef03-46a9-b7d0-b1d45ea79b65 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.041823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bbf26-ef03-46a9-b7d0-b1d45ea79b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.042080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bbf26-ef03-46a9-b7d0-b1d45ea79b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.042748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bbf26-ef03-46a9-b7d0-b1d45ea79b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.044663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bbf26-ef03-46a9-b7d0-b1d45ea79b65 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.074405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 629/1258] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/projects => generated 389 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:40.075614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bbf26-ef03-46a9-b7d0-b1d45ea79b65 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.082738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa656b7d-0876-4e8e-a83b-d801850b5a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.083408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa656b7d-0876-4e8e-a83b-d801850b5a68 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.083765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa656b7d-0876-4e8e-a83b-d801850b5a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.084211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa656b7d-0876-4e8e-a83b-d801850b5a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.084838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa656b7d-0876-4e8e-a83b-d801850b5a68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.086817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa656b7d-0876-4e8e-a83b-d801850b5a68 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.117458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 630/1259] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:39 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:40.117458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa656b7d-0876-4e8e-a83b-d801850b5a68 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.134368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:40.136971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:40.148340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bbf26-ef03-46a9-b7d0-b1d45ea79b65 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.178037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa656b7d-0876-4e8e-a83b-d801850b5a68 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.182809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0851a-31f1-4dff-9304-f687df355872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.183425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0851a-31f1-4dff-9304-f687df355872 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.183704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0851a-31f1-4dff-9304-f687df355872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.184003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0851a-31f1-4dff-9304-f687df355872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.184673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0851a-31f1-4dff-9304-f687df355872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.186313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0851a-31f1-4dff-9304-f687df355872 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.196009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 631/1260] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.199033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0851a-31f1-4dff-9304-f687df355872 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.205237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf89e7-a96f-4242-b448-64e62aafe82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.205955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf89e7-a96f-4242-b448-64e62aafe82e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.206268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf89e7-a96f-4242-b448-64e62aafe82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.206541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf89e7-a96f-4242-b448-64e62aafe82e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.207228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf89e7-a96f-4242-b448-64e62aafe82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.208743 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0851a-31f1-4dff-9304-f687df355872 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.208871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf89e7-a96f-4242-b448-64e62aafe82e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.213258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efacf40-cf65-4b47-af0a-f97c786ddede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.213898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efacf40-cf65-4b47-af0a-f97c786ddede admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.214211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efacf40-cf65-4b47-af0a-f97c786ddede admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.214488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efacf40-cf65-4b47-af0a-f97c786ddede admin admin] PATH_INFO: `/v3/projects/7402a850a25a48c48faf88fb4f3a56ae/users/0e8717d368d44102bea0eb82fbee8a5c/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.222637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 630/1261] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.222637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf89e7-a96f-4242-b448-64e62aafe82e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.230223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efacf40-cf65-4b47-af0a-f97c786ddede admin admin] RBAC: Authorizing `identity:create_grant(project_id=7402a850a25a48c48faf88fb4f3a56ae, user_id=0e8717d368d44102bea0eb82fbee8a5c, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.230449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf89e7-a96f-4242-b448-64e62aafe82e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.232879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efacf40-cf65-4b47-af0a-f97c786ddede admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.235061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8f88-4f5e-437f-8232-fcf2fc22dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.235774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8f88-4f5e-437f-8232-fcf2fc22dde4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.236067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8f88-4f5e-437f-8232-fcf2fc22dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.236332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8f88-4f5e-437f-8232-fcf2fc22dde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.236963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8f88-4f5e-437f-8232-fcf2fc22dde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.238672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8f88-4f5e-437f-8232-fcf2fc22dde4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.251293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 632/1262] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:40 2022] PUT /identity/v3/projects/7402a850a25a48c48faf88fb4f3a56ae/users/0e8717d368d44102bea0eb82fbee8a5c/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:40.251293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60e7fb-84e7-4a51-8367-503e33b2f4d5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.251293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60e7fb-84e7-4a51-8367-503e33b2f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.251293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60e7fb-84e7-4a51-8367-503e33b2f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.251802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 631/1263] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.251802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8f88-4f5e-437f-8232-fcf2fc22dde4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.262782 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8f88-4f5e-437f-8232-fcf2fc22dde4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.280952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2a60e7fb-84e7-4a51-8367-503e33b2f4d5 None None] MFA Rules not processed for user `ec0e6cbf12de4edb9d3ab4ab0d2448b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:40.282347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2a60e7fb-84e7-4a51-8367-503e33b2f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:40.319757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc2390-0bf3-4f62-87e2-2f438146f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.320378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc2390-0bf3-4f62-87e2-2f438146f973 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.320664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc2390-0bf3-4f62-87e2-2f438146f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.320926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc2390-0bf3-4f62-87e2-2f438146f973 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.321559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc2390-0bf3-4f62-87e2-2f438146f973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.323560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc2390-0bf3-4f62-87e2-2f438146f973 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.346635 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 633/1264] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:40.347230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 632/1265] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:40 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:40.348121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60e7fb-84e7-4a51-8367-503e33b2f4d5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.348272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc2390-0bf3-4f62-87e2-2f438146f973 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.362293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60e7fb-84e7-4a51-8367-503e33b2f4d5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.363441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc2390-0bf3-4f62-87e2-2f438146f973 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.367205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bfe33-05cd-45ee-8bd2-e302408d5d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.367823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bfe33-05cd-45ee-8bd2-e302408d5d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.368097 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bfe33-05cd-45ee-8bd2-e302408d5d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.368454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bfe33-05cd-45ee-8bd2-e302408d5d32 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.369093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274bfe33-05cd-45ee-8bd2-e302408d5d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.369512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee4b45-4e4d-438b-b422-27b1e705442e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.370337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4b45-4e4d-438b-b422-27b1e705442e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.370993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4b45-4e4d-438b-b422-27b1e705442e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.371200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274bfe33-05cd-45ee-8bd2-e302408d5d32 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.371631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4b45-4e4d-438b-b422-27b1e705442e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.372488 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee4b45-4e4d-438b-b422-27b1e705442e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.375059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee4b45-4e4d-438b-b422-27b1e705442e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.398809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 634/1266] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:40 2022] POST /identity/v3/users => generated 295 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:40.398809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bfe33-05cd-45ee-8bd2-e302408d5d32 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.402558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 633/1267] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:40 2022] POST /identity/v3/users => generated 292 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:40.403493 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee4b45-4e4d-438b-b422-27b1e705442e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.405981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bfe33-05cd-45ee-8bd2-e302408d5d32 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.410841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d00ea-1429-465e-aa08-ae39dd56a982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.411438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d00ea-1429-465e-aa08-ae39dd56a982 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.411719 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d00ea-1429-465e-aa08-ae39dd56a982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.411986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d00ea-1429-465e-aa08-ae39dd56a982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.412621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d00ea-1429-465e-aa08-ae39dd56a982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.414254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d00ea-1429-465e-aa08-ae39dd56a982 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.415247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee4b45-4e4d-438b-b422-27b1e705442e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.420014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5d7ed-ef3d-4044-9e73-2f0c7cd86116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.420648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5d7ed-ef3d-4044-9e73-2f0c7cd86116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.420922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5d7ed-ef3d-4044-9e73-2f0c7cd86116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.421219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5d7ed-ef3d-4044-9e73-2f0c7cd86116 admin admin] PATH_INFO: `/v3/projects/7150594e44c241a6a4a6110b0a676cc0/users/4f9775dfc80e4aaaba521a699eb3f664/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.428052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 635/1268] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.428052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704e5da-7df5-4de3-ad94-2a5d087aaa56 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.428052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704e5da-7df5-4de3-ad94-2a5d087aaa56 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.428052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704e5da-7df5-4de3-ad94-2a5d087aaa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.436311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5d7ed-ef3d-4044-9e73-2f0c7cd86116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7150594e44c241a6a4a6110b0a676cc0, user_id=4f9775dfc80e4aaaba521a699eb3f664, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.439885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5d7ed-ef3d-4044-9e73-2f0c7cd86116 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.445604 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4704e5da-7df5-4de3-ad94-2a5d087aaa56 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:40.447000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4704e5da-7df5-4de3-ad94-2a5d087aaa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:40.453961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 634/1269] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:40 2022] PUT /identity/v3/projects/7150594e44c241a6a4a6110b0a676cc0/users/4f9775dfc80e4aaaba521a699eb3f664/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:40.454786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5d7ed-ef3d-4044-9e73-2f0c7cd86116 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.492466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:40.494500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:40.508656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 636/1270] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:40.509317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704e5da-7df5-4de3-ad94-2a5d087aaa56 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.518667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704e5da-7df5-4de3-ad94-2a5d087aaa56 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.523193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e338435-bb4f-4ca4-868f-163860812439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.523784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338435-bb4f-4ca4-868f-163860812439 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.524503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338435-bb4f-4ca4-868f-163860812439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.524503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338435-bb4f-4ca4-868f-163860812439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.524961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e338435-bb4f-4ca4-868f-163860812439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.526611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e338435-bb4f-4ca4-868f-163860812439 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.528343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5d7ed-ef3d-4044-9e73-2f0c7cd86116 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.533056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f412020-4c46-4a92-8e62-9e7abb497f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.533891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412020-4c46-4a92-8e62-9e7abb497f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.534194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412020-4c46-4a92-8e62-9e7abb497f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.534480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412020-4c46-4a92-8e62-9e7abb497f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.535118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f412020-4c46-4a92-8e62-9e7abb497f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.536694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f412020-4c46-4a92-8e62-9e7abb497f2e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.536892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 637/1271] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.538080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b587f-65a0-4769-808b-37d8ebb4e541 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.538385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b587f-65a0-4769-808b-37d8ebb4e541 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.538660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b587f-65a0-4769-808b-37d8ebb4e541 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.540043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 638/1272] 10.222.0.25 () {54 vars in 961 bytes} [Wed Nov 2 07:27:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:40.540533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b587f-65a0-4769-808b-37d8ebb4e541 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.552780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b587f-65a0-4769-808b-37d8ebb4e541 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.556798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 635/1273] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.556798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f412020-4c46-4a92-8e62-9e7abb497f2e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.560802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12d780-e9d0-45e4-86c2-c21629ebdf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.561399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12d780-e9d0-45e4-86c2-c21629ebdf0b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.561678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12d780-e9d0-45e4-86c2-c21629ebdf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.561952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12d780-e9d0-45e4-86c2-c21629ebdf0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.562626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12d780-e9d0-45e4-86c2-c21629ebdf0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.564285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12d780-e9d0-45e4-86c2-c21629ebdf0b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.564414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f412020-4c46-4a92-8e62-9e7abb497f2e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.569089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906dcf3-e432-47bd-89a1-59b3f8b14e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.569685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906dcf3-e432-47bd-89a1-59b3f8b14e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.569967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906dcf3-e432-47bd-89a1-59b3f8b14e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.570270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906dcf3-e432-47bd-89a1-59b3f8b14e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.570904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906dcf3-e432-47bd-89a1-59b3f8b14e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.572535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906dcf3-e432-47bd-89a1-59b3f8b14e73 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.574709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 639/1274] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.575315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12d780-e9d0-45e4-86c2-c21629ebdf0b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.582909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 636/1275] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.584137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383634e-007f-4eeb-b250-c4b3bf172756 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.584420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383634e-007f-4eeb-b250-c4b3bf172756 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.584691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383634e-007f-4eeb-b250-c4b3bf172756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.588195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12d780-e9d0-45e4-86c2-c21629ebdf0b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.592834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294d03f-7a9a-4005-939d-4c36b38b5bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.593843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294d03f-7a9a-4005-939d-4c36b38b5bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.594153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294d03f-7a9a-4005-939d-4c36b38b5bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.594454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294d03f-7a9a-4005-939d-4c36b38b5bd5 admin admin] PATH_INFO: `/v3/projects/07d6ae57e56143f5b8755cec0baf10e6/users/20ad7a926bb846b5b61e40864a9d7707/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.609209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294d03f-7a9a-4005-939d-4c36b38b5bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d6ae57e56143f5b8755cec0baf10e6, user_id=20ad7a926bb846b5b61e40864a9d7707, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.612044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294d03f-7a9a-4005-939d-4c36b38b5bd5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.612520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4383634e-007f-4eeb-b250-c4b3bf172756 None None] MFA Rules not processed for user `4f9775dfc80e4aaaba521a699eb3f664`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:40.613735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4383634e-007f-4eeb-b250-c4b3bf172756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:40.625602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 640/1276] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:40 2022] PUT /identity/v3/projects/07d6ae57e56143f5b8755cec0baf10e6/users/20ad7a926bb846b5b61e40864a9d7707/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:40.627173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294d03f-7a9a-4005-939d-4c36b38b5bd5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.677732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 637/1277] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:40.678646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383634e-007f-4eeb-b250-c4b3bf172756 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.689072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383634e-007f-4eeb-b250-c4b3bf172756 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.695938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294d03f-7a9a-4005-939d-4c36b38b5bd5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.700088 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15ce45-780c-44ed-a16a-eece79cea207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.700729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15ce45-780c-44ed-a16a-eece79cea207 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.701003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15ce45-780c-44ed-a16a-eece79cea207 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.701266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15ce45-780c-44ed-a16a-eece79cea207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.747480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee707f-c1e6-4b7a-9046-74f404a06623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.748370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee707f-c1e6-4b7a-9046-74f404a06623 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.748649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee707f-c1e6-4b7a-9046-74f404a06623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.748924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee707f-c1e6-4b7a-9046-74f404a06623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.749729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee707f-c1e6-4b7a-9046-74f404a06623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.751808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee707f-c1e6-4b7a-9046-74f404a06623 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.761609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 638/1278] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.764923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15ce45-780c-44ed-a16a-eece79cea207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.766012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d760-9ab7-488b-99c1-5c72f8f0a676 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.766327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d760-9ab7-488b-99c1-5c72f8f0a676 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.766423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15ce45-780c-44ed-a16a-eece79cea207 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.766600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d760-9ab7-488b-99c1-5c72f8f0a676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.777332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 641/1279] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:40.778319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15ce45-780c-44ed-a16a-eece79cea207 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.788354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15ce45-780c-44ed-a16a-eece79cea207 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.792916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315533a-ee43-4792-beea-9e5a3c4bca91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.793572 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315533a-ee43-4792-beea-9e5a3c4bca91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.793846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315533a-ee43-4792-beea-9e5a3c4bca91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.794141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315533a-ee43-4792-beea-9e5a3c4bca91 admin admin] PATH_INFO: `/v3/projects/a3c154cdcab34bd4bc2b5c0e198ccaa1/users/8f151dc283e8498f961b49320130f770/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.805736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a694d760-9ab7-488b-99c1-5c72f8f0a676 None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:40.807091 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a694d760-9ab7-488b-99c1-5c72f8f0a676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:40.811673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315533a-ee43-4792-beea-9e5a3c4bca91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c154cdcab34bd4bc2b5c0e198ccaa1, user_id=8f151dc283e8498f961b49320130f770, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.816968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315533a-ee43-4792-beea-9e5a3c4bca91 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.837509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:40.840035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 642/1280] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:40 2022] PUT /identity/v3/projects/a3c154cdcab34bd4bc2b5c0e198ccaa1/users/8f151dc283e8498f961b49320130f770/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:40.841602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315533a-ee43-4792-beea-9e5a3c4bca91 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.890791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 639/1281] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Nov 2 07:27:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:40.891770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694d760-9ab7-488b-99c1-5c72f8f0a676 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:40.905956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694d760-9ab7-488b-99c1-5c72f8f0a676 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.915535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:40.964328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315533a-ee43-4792-beea-9e5a3c4bca91 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:40.973367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc52fc-b240-42d3-925a-7feb6d2c7f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.974510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc52fc-b240-42d3-925a-7feb6d2c7f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.975073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc52fc-b240-42d3-925a-7feb6d2c7f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.975522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc52fc-b240-42d3-925a-7feb6d2c7f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.976571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc52fc-b240-42d3-925a-7feb6d2c7f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.978870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc52fc-b240-42d3-925a-7feb6d2c7f8d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.981341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffdffa-4191-4e62-9f91-f31fd839e707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:40.982311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffdffa-4191-4e62-9f91-f31fd839e707 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.982587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffdffa-4191-4e62-9f91-f31fd839e707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.982857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffdffa-4191-4e62-9f91-f31fd839e707 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:40.983494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffdffa-4191-4e62-9f91-f31fd839e707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:40.985427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffdffa-4191-4e62-9f91-f31fd839e707 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:40.994427 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 643/1282] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:40.996172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a38f3-4cef-400f-af15-fe480e028a6e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:40.996584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a38f3-4cef-400f-af15-fe480e028a6e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:40.997130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a38f3-4cef-400f-af15-fe480e028a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.001017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 640/1283] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:41.005812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba19692-fc0e-454b-88a0-979e560ec156 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.006213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba19692-fc0e-454b-88a0-979e560ec156 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.006392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba19692-fc0e-454b-88a0-979e560ec156 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.007670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 641/1284] 10.222.0.25 () {54 vars in 961 bytes} [Wed Nov 2 07:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:41.013948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba19692-fc0e-454b-88a0-979e560ec156 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.031853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba19692-fc0e-454b-88a0-979e560ec156 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.033329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8b8a38f3-4cef-400f-af15-fe480e028a6e None None] MFA Rules not processed for user `20ad7a926bb846b5b61e40864a9d7707`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:41.034983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8b8a38f3-4cef-400f-af15-fe480e028a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:41.037521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d6ccc-1df0-4b56-b7dc-db4cc9d97de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.038609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d6ccc-1df0-4b56-b7dc-db4cc9d97de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.038945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d6ccc-1df0-4b56-b7dc-db4cc9d97de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.039322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d6ccc-1df0-4b56-b7dc-db4cc9d97de5 admin admin] PATH_INFO: `/v3/projects/7c651864ae8c4026920a713b4a7d9a33/users/5318b31b1138460a8ce0a705b0b720a0/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.057541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d6ccc-1df0-4b56-b7dc-db4cc9d97de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c651864ae8c4026920a713b4a7d9a33, user_id=5318b31b1138460a8ce0a705b0b720a0, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:41.060334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d6ccc-1df0-4b56-b7dc-db4cc9d97de5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:41.075183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 642/1285] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:41 2022] PUT /identity/v3/projects/7c651864ae8c4026920a713b4a7d9a33/users/5318b31b1138460a8ce0a705b0b720a0/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:41.076245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65c729-a1ec-43e4-8b99-7e7c47b4148f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.076532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65c729-a1ec-43e4-8b99-7e7c47b4148f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.076801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65c729-a1ec-43e4-8b99-7e7c47b4148f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.102399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7c65c729-a1ec-43e4-8b99-7e7c47b4148f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:41.103642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7c65c729-a1ec-43e4-8b99-7e7c47b4148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:41.133909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 644/1286] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:41.133909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a38f3-4cef-400f-af15-fe480e028a6e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.169243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 643/1287] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:41.179890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65c729-a1ec-43e4-8b99-7e7c47b4148f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.189948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65c729-a1ec-43e4-8b99-7e7c47b4148f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.194700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1e020-bb20-42ea-9b2c-4bbabdf3e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.195558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1e020-bb20-42ea-9b2c-4bbabdf3e3de admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.195843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1e020-bb20-42ea-9b2c-4bbabdf3e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.196118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1e020-bb20-42ea-9b2c-4bbabdf3e3de admin admin] PATH_INFO: `/v3/projects/80e2c88e09a546bdbe9e644295a237bc/users/006ae080f2884719bd4ddabe815d0d01/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.217588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1e020-bb20-42ea-9b2c-4bbabdf3e3de admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e2c88e09a546bdbe9e644295a237bc, user_id=006ae080f2884719bd4ddabe815d0d01, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:41.221059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1e020-bb20-42ea-9b2c-4bbabdf3e3de admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:41.237083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a38f3-4cef-400f-af15-fe480e028a6e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.241052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 644/1288] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:41 2022] PUT /identity/v3/projects/80e2c88e09a546bdbe9e644295a237bc/users/006ae080f2884719bd4ddabe815d0d01/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:41.242063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1e020-bb20-42ea-9b2c-4bbabdf3e3de admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.272506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:41.311735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:41.327580 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a9501-ca41-4116-9729-64000bc8a199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.328240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a9501-ca41-4116-9729-64000bc8a199 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.328516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a9501-ca41-4116-9729-64000bc8a199 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.328797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a9501-ca41-4116-9729-64000bc8a199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.333981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1e020-bb20-42ea-9b2c-4bbabdf3e3de admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.338711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71e6e3-57f5-42cf-82af-9fbc808f4e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.339382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71e6e3-57f5-42cf-82af-9fbc808f4e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.339656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71e6e3-57f5-42cf-82af-9fbc808f4e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.339930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71e6e3-57f5-42cf-82af-9fbc808f4e4b admin admin] PATH_INFO: `/v3/projects/c1d95dd88d4c44e1a251604ef3a493ba/users/af51aa4023f949338cb0033af92a7c2b/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.356287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a71e6e3-57f5-42cf-82af-9fbc808f4e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d95dd88d4c44e1a251604ef3a493ba, user_id=af51aa4023f949338cb0033af92a7c2b, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:41.358972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a71e6e3-57f5-42cf-82af-9fbc808f4e4b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:41.376385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 645/1289] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:41 2022] PUT /identity/v3/projects/c1d95dd88d4c44e1a251604ef3a493ba/users/af51aa4023f949338cb0033af92a7c2b/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:41.377855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543708f-6387-4518-b4e6-7182e7996f51 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.378207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543708f-6387-4518-b4e6-7182e7996f51 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.378518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543708f-6387-4518-b4e6-7182e7996f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.398961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d543708f-6387-4518-b4e6-7182e7996f51 None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:41.400162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d543708f-6387-4518-b4e6-7182e7996f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:41.406262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a9501-ca41-4116-9729-64000bc8a199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:41.407888 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a9501-ca41-4116-9729-64000bc8a199 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:41.417491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 645/1290] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:41.418913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453e57-13d0-4eb9-9e1f-903df31704dc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.419199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453e57-13d0-4eb9-9e1f-903df31704dc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.419577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453e57-13d0-4eb9-9e1f-903df31704dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.439593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d4453e57-13d0-4eb9-9e1f-903df31704dc None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:41.440654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d4453e57-13d0-4eb9-9e1f-903df31704dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:41.491575 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 646/1291] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Nov 2 07:27:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:41.492804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543708f-6387-4518-b4e6-7182e7996f51 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.520357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 646/1292] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:41.521313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4453e57-13d0-4eb9-9e1f-903df31704dc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.571023 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543708f-6387-4518-b4e6-7182e7996f51 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.577719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c135ef-e666-4604-a328-9f623e54a66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.578425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c135ef-e666-4604-a328-9f623e54a66c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.578712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c135ef-e666-4604-a328-9f623e54a66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.578978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c135ef-e666-4604-a328-9f623e54a66c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.579686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c135ef-e666-4604-a328-9f623e54a66c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:41.582756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c135ef-e666-4604-a328-9f623e54a66c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:41.602289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4453e57-13d0-4eb9-9e1f-903df31704dc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.610382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c894f-9d03-49af-a1fa-a29773c5cf0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.611153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c894f-9d03-49af-a1fa-a29773c5cf0b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.611541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c894f-9d03-49af-a1fa-a29773c5cf0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.611923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c894f-9d03-49af-a1fa-a29773c5cf0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.619499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 647/1293] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:41 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:41.619499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fa897-8a79-4c7c-860e-c07a090d83e8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.619499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fa897-8a79-4c7c-860e-c07a090d83e8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.619499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fa897-8a79-4c7c-860e-c07a090d83e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.642016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cc2fa897-8a79-4c7c-860e-c07a090d83e8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:41.643267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c894f-9d03-49af-a1fa-a29773c5cf0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:41.643510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc2fa897-8a79-4c7c-860e-c07a090d83e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:41.645213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c894f-9d03-49af-a1fa-a29773c5cf0b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:41.650294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:41.670298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 647/1294] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:41.673801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed97d-5ca8-4da3-b91c-08a3fd1519d1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.674082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed97d-5ca8-4da3-b91c-08a3fd1519d1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.675813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed97d-5ca8-4da3-b91c-08a3fd1519d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.725002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2b0ed97d-5ca8-4da3-b91c-08a3fd1519d1 None None] MFA Rules not processed for user `af51aa4023f949338cb0033af92a7c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:41.727332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2b0ed97d-5ca8-4da3-b91c-08a3fd1519d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:41.765279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 648/1295] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:41.766349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fa897-8a79-4c7c-860e-c07a090d83e8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.786413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:41.877736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 648/1296] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:41.879050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ed97d-5ca8-4da3-b91c-08a3fd1519d1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:41.942285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fa897-8a79-4c7c-860e-c07a090d83e8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.955231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ed97d-5ca8-4da3-b91c-08a3fd1519d1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:41.960190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bf249-8f69-4feb-9578-3c4b50a69a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.960681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bf249-8f69-4feb-9578-3c4b50a69a45 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.961092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bf249-8f69-4feb-9578-3c4b50a69a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.961240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bf249-8f69-4feb-9578-3c4b50a69a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.961809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cc9a9-0eb1-427b-8f2b-ce301efd2a8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:41.963358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cc9a9-0eb1-427b-8f2b-ce301efd2a8f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:41.964957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cc9a9-0eb1-427b-8f2b-ce301efd2a8f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:41.965244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cc9a9-0eb1-427b-8f2b-ce301efd2a8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:41.974600 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48bf249-8f69-4feb-9578-3c4b50a69a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:41.976324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48bf249-8f69-4feb-9578-3c4b50a69a45 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:41.991593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 649/1297] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:41.992738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bf249-8f69-4feb-9578-3c4b50a69a45 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.005378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bf249-8f69-4feb-9578-3c4b50a69a45 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.011772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42d3e-1c76-49a6-971e-9ba4e91a9e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.012670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42d3e-1c76-49a6-971e-9ba4e91a9e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.013127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42d3e-1c76-49a6-971e-9ba4e91a9e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.013538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42d3e-1c76-49a6-971e-9ba4e91a9e86 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.014678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42d3e-1c76-49a6-971e-9ba4e91a9e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.018008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42d3e-1c76-49a6-971e-9ba4e91a9e86 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.069197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 650/1298] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:41 2022] POST /identity/v3/users => generated 299 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:42.070391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42d3e-1c76-49a6-971e-9ba4e91a9e86 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.091175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42d3e-1c76-49a6-971e-9ba4e91a9e86 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.098868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cc9a9-0eb1-427b-8f2b-ce301efd2a8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.100507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a68cff-70d5-42b5-8c05-130e0d581cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.101204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68cff-70d5-42b5-8c05-130e0d581cbb service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.101483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68cff-70d5-42b5-8c05-130e0d581cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.101826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68cff-70d5-42b5-8c05-130e0d581cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.102675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cc9a9-0eb1-427b-8f2b-ce301efd2a8f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.115635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 649/1299] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:42.121812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ebeaa-1826-419d-9190-110f07bb5de0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.122153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ebeaa-1826-419d-9190-110f07bb5de0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.122693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ebeaa-1826-419d-9190-110f07bb5de0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.130083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a68cff-70d5-42b5-8c05-130e0d581cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.131393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 650/1300] 10.222.0.25 () {54 vars in 912 bytes} [Wed Nov 2 07:27:42 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:42.132540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ebeaa-1826-419d-9190-110f07bb5de0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.135971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a68cff-70d5-42b5-8c05-130e0d581cbb service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.148786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ebeaa-1826-419d-9190-110f07bb5de0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.155787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8634e9-ea1e-4fee-acd8-df95ccae81cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.156405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8634e9-ea1e-4fee-acd8-df95ccae81cf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.156730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8634e9-ea1e-4fee-acd8-df95ccae81cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.157051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8634e9-ea1e-4fee-acd8-df95ccae81cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.158027 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8634e9-ea1e-4fee-acd8-df95ccae81cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.160149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8634e9-ea1e-4fee-acd8-df95ccae81cf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.164417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 651/1301] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:42.168037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85fe41-42cc-46b7-9e62-d925a5a2ddd3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.168330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85fe41-42cc-46b7-9e62-d925a5a2ddd3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.169786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85fe41-42cc-46b7-9e62-d925a5a2ddd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.174960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 651/1302] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:42.178878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8634e9-ea1e-4fee-acd8-df95ccae81cf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.190882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8634e9-ea1e-4fee-acd8-df95ccae81cf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.197984 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acecac2-401d-457d-bd7e-8d53fe779338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.199074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acecac2-401d-457d-bd7e-8d53fe779338 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.199492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acecac2-401d-457d-bd7e-8d53fe779338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.200040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acecac2-401d-457d-bd7e-8d53fe779338 admin admin] PATH_INFO: `/v3/projects/66b62e7dcf4a4d4e9ba8f7104e06f2fc/users/15782d6520214a368d7155e5d63dae87/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.208963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fa85fe41-42cc-46b7-9e62-d925a5a2ddd3 None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:42.220489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fa85fe41-42cc-46b7-9e62-d925a5a2ddd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:42.221419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acecac2-401d-457d-bd7e-8d53fe779338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b62e7dcf4a4d4e9ba8f7104e06f2fc, user_id=15782d6520214a368d7155e5d63dae87, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.224787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acecac2-401d-457d-bd7e-8d53fe779338 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.246264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 652/1303] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:27:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:42.254163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 652/1304] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:42 2022] PUT /identity/v3/projects/66b62e7dcf4a4d4e9ba8f7104e06f2fc/users/15782d6520214a368d7155e5d63dae87/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:27:42.257819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85fe41-42cc-46b7-9e62-d925a5a2ddd3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.262686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acecac2-401d-457d-bd7e-8d53fe779338 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.272990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85fe41-42cc-46b7-9e62-d925a5a2ddd3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.280204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:42.349229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acecac2-401d-457d-bd7e-8d53fe779338 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.356699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad8e7-c9d4-49a8-90fd-ec5a67a0ceb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.361435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad8e7-c9d4-49a8-90fd-ec5a67a0ceb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.362931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad8e7-c9d4-49a8-90fd-ec5a67a0ceb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.363453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad8e7-c9d4-49a8-90fd-ec5a67a0ceb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.404214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6ba8b-f856-4c12-9c21-05277452969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.404856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6ba8b-f856-4c12-9c21-05277452969c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.405275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6ba8b-f856-4c12-9c21-05277452969c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.405557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6ba8b-f856-4c12-9c21-05277452969c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.406244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6ba8b-f856-4c12-9c21-05277452969c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.407918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6ba8b-f856-4c12-9c21-05277452969c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.439722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 653/1305] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:42.444495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6ba8b-f856-4c12-9c21-05277452969c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.462487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6ba8b-f856-4c12-9c21-05277452969c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.469016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248330d-10fe-4846-9527-be89960d3e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.469881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248330d-10fe-4846-9527-be89960d3e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.470485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248330d-10fe-4846-9527-be89960d3e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.470884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248330d-10fe-4846-9527-be89960d3e1f admin admin] PATH_INFO: `/v3/projects/66b62e7dcf4a4d4e9ba8f7104e06f2fc/users/15782d6520214a368d7155e5d63dae87/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.473489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248330d-10fe-4846-9527-be89960d3e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b62e7dcf4a4d4e9ba8f7104e06f2fc, user_id=15782d6520214a368d7155e5d63dae87, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.476851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248330d-10fe-4846-9527-be89960d3e1f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.490666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ad8e7-c9d4-49a8-90fd-ec5a67a0ceb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.491140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 654/1306] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:42 2022] PUT /identity/v3/projects/66b62e7dcf4a4d4e9ba8f7104e06f2fc/users/15782d6520214a368d7155e5d63dae87/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:42.492599 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2fbb-fba4-410c-8e0f-392337cf54d6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.492957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ad8e7-c9d4-49a8-90fd-ec5a67a0ceb9 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.493096 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2fbb-fba4-410c-8e0f-392337cf54d6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.493431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2fbb-fba4-410c-8e0f-392337cf54d6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.495070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 655/1307] 10.222.0.25 () {54 vars in 912 bytes} [Wed Nov 2 07:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:42.500811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99c237-d641-4ad6-a8e0-d4101f419721 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.501244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99c237-d641-4ad6-a8e0-d4101f419721 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.501605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99c237-d641-4ad6-a8e0-d4101f419721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.513730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 653/1308] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:42.516012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eaaa2-991c-4956-95c9-2d028c961aec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.516542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eaaa2-991c-4956-95c9-2d028c961aec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.517184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eaaa2-991c-4956-95c9-2d028c961aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.547082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4b9eaaa2-991c-4956-95c9-2d028c961aec None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:42.548623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4b9eaaa2-991c-4956-95c9-2d028c961aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:42.618677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:42.627463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4f99c237-d641-4ad6-a8e0-d4101f419721 None None] MFA Rules not processed for user `15782d6520214a368d7155e5d63dae87`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:42.630068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4f99c237-d641-4ad6-a8e0-d4101f419721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:42.695091 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 654/1309] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:27:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:42.706611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eaaa2-991c-4956-95c9-2d028c961aec None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.778637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 656/1310] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:42.811595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99c237-d641-4ad6-a8e0-d4101f419721 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.819941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:42.828467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99c237-d641-4ad6-a8e0-d4101f419721 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.834542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eaaa2-991c-4956-95c9-2d028c961aec None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.845368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513e49-9419-4798-a4a3-792955ad8b3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.846434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513e49-9419-4798-a4a3-792955ad8b3d service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.847433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513e49-9419-4798-a4a3-792955ad8b3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.848171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601f584-4db4-4496-8b1e-7f51c172604e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.848866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601f584-4db4-4496-8b1e-7f51c172604e service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.849293 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513e49-9419-4798-a4a3-792955ad8b3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.849658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601f584-4db4-4496-8b1e-7f51c172604e service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.850598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601f584-4db4-4496-8b1e-7f51c172604e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.873006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513e49-9419-4798-a4a3-792955ad8b3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.875413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513e49-9419-4798-a4a3-792955ad8b3d service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.902496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 655/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:42.904626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513e49-9419-4798-a4a3-792955ad8b3d service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:42.923922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513e49-9419-4798-a4a3-792955ad8b3d service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:42.937248 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c12f0-9751-4b53-95e1-4eda5c1393d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:42.937870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c12f0-9751-4b53-95e1-4eda5c1393d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:42.938176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c12f0-9751-4b53-95e1-4eda5c1393d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:42.938461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c12f0-9751-4b53-95e1-4eda5c1393d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:42.960188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c12f0-9751-4b53-95e1-4eda5c1393d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.961658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c12f0-9751-4b53-95e1-4eda5c1393d6 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.982107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 656/1312] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:42.984340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601f584-4db4-4496-8b1e-7f51c172604e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:42.986068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601f584-4db4-4496-8b1e-7f51c172604e service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:42.997988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 657/1313] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:43.832713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c12f0-9751-4b53-95e1-4eda5c1393d6 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:44.034738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c12f0-9751-4b53-95e1-4eda5c1393d6 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:44.050890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:44.051553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:44.051965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:44.052240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] PATH_INFO: `/v3/users/20ad7a926bb846b5b61e40864a9d7707` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:44.055086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ad7a926bb846b5b61e40864a9d7707)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:44.056272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:44.056272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 07:27:44.056272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: Nov 02 07:27:44.059299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:44.255537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ad7a926bb846b5b61e40864a9d7707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.264627 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ad7a926bb846b5b61e40864a9d7707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.270802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601f584-4db4-4496-8b1e-7f51c172604e service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:44.287768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601f584-4db4-4496-8b1e-7f51c172604e service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:44.293311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:44.294075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:44.294880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:44.297620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] PATH_INFO: `/v3/users/ec0e6cbf12de4edb9d3ab4ab0d2448b3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:44.307550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0e6cbf12de4edb9d3ab4ab0d2448b3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:44.308838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:44.351197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ad7a926bb846b5b61e40864a9d7707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.377902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ad7a926bb846b5b61e40864a9d7707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.378903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-957e1e06-628a-4621-9fbe-82fb85dd3c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ad7a926bb846b5b61e40864a9d7707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.403649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 657/1314] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:43 2022] DELETE /identity/v3/users/20ad7a926bb846b5b61e40864a9d7707 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:44.406632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0e6cbf12de4edb9d3ab4ab0d2448b3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.411324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73214f6-9749-49c4-ac1c-8b51d09caf7b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:44.411324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73214f6-9749-49c4-ac1c-8b51d09caf7b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:44.411324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73214f6-9749-49c4-ac1c-8b51d09caf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:44.412453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0e6cbf12de4edb9d3ab4ab0d2448b3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.436715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0e6cbf12de4edb9d3ab4ab0d2448b3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.448540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0e6cbf12de4edb9d3ab4ab0d2448b3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.450105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a73214f6-9749-49c4-ac1c-8b51d09caf7b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:44.452748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-13b4e78d-5741-43d9-b79c-c4eeeebfef53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0e6cbf12de4edb9d3ab4ab0d2448b3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:44.453306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a73214f6-9749-49c4-ac1c-8b51d09caf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:44.478296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 658/1315] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:44 2022] DELETE /identity/v3/users/ec0e6cbf12de4edb9d3ab4ab0d2448b3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:44.482979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752034-89b0-4dc5-9d51-f1ebff92ca5a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:44.483326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752034-89b0-4dc5-9d51-f1ebff92ca5a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:44.483671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752034-89b0-4dc5-9d51-f1ebff92ca5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:44.525083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-79752034-89b0-4dc5-9d51-f1ebff92ca5a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:44.526711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-79752034-89b0-4dc5-9d51-f1ebff92ca5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:44.532059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:44.562138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:44.583359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 658/1316] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:44.594646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73214f6-9749-49c4-ac1c-8b51d09caf7b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:44.597983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a73214f6-9749-49c4-ac1c-8b51d09caf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:44.706222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73214f6-9749-49c4-ac1c-8b51d09caf7b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:44.711657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 659/1317] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:44.779909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:44.780626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:44.781033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:44.781368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:44.786145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752034-89b0-4dc5-9d51-f1ebff92ca5a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:44.795828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:44.796807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752034-89b0-4dc5-9d51-f1ebff92ca5a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:44.797554 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:44.810154 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de0595-39d2-4453-ab33-bbd20f29b945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:44.810798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de0595-39d2-4453-ab33-bbd20f29b945 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:44.811222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de0595-39d2-4453-ab33-bbd20f29b945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:44.811505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de0595-39d2-4453-ab33-bbd20f29b945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:44.814478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 659/1318] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:44.823922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de0595-39d2-4453-ab33-bbd20f29b945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:44.826544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de0595-39d2-4453-ab33-bbd20f29b945 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:44.838560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 660/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:46.466683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:46.469798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:46.506135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8295f-8f74-42a4-b6dd-3084f5c54075 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:46.520990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:46.522819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:46.523413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:46.524215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] PATH_INFO: `/v3/projects/07d6ae57e56143f5b8755cec0baf10e6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:46.531988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d6ae57e56143f5b8755cec0baf10e6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:46.538829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:46.649890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d6ae57e56143f5b8755cec0baf10e6', 'request_id': 'req-90af16a7-7f46-4470-8fc1-528bb58b5edf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:46.675387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d6ae57e56143f5b8755cec0baf10e6', 'request_id': 'req-90af16a7-7f46-4470-8fc1-528bb58b5edf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:46.695636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d6ae57e56143f5b8755cec0baf10e6', 'request_id': 'req-90af16a7-7f46-4470-8fc1-528bb58b5edf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:46.726523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] The token cache is being invalidate because project 07d6ae57e56143f5b8755cec0baf10e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:46.726984 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:46.727794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:46.734468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 660/1320] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:46 2022] DELETE /identity/v3/projects/07d6ae57e56143f5b8755cec0baf10e6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:46.735733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de0595-39d2-4453-ab33-bbd20f29b945 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:46.749565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de0595-39d2-4453-ab33-bbd20f29b945 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:46.994708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:46.995506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.000608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.000769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] PATH_INFO: `/v3/users/15782d6520214a368d7155e5d63dae87` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.002668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] RBAC: Authorizing `identity:delete_user(user_id=15782d6520214a368d7155e5d63dae87)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:47.008121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:47.166186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15782d6520214a368d7155e5d63dae87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.166186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15782d6520214a368d7155e5d63dae87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.192251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15782d6520214a368d7155e5d63dae87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.200621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:47.202537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15782d6520214a368d7155e5d63dae87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.203706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:47.204967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15782d6520214a368d7155e5d63dae87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.225642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af16a7-7f46-4470-8fc1-528bb58b5edf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:47.231480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:47.232670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.233156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.233619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] PATH_INFO: `/v3/projects/02351bab6e5041a7969b50e93fa7d010` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.234225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 661/1321] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:46 2022] DELETE /identity/v3/users/15782d6520214a368d7155e5d63dae87 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:27:47.235499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] RBAC: Authorizing `identity:delete_project(project_id=02351bab6e5041a7969b50e93fa7d010)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:47.235623 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:47.237915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:47.238340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:47.262684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:47.297539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:47.315505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02351bab6e5041a7969b50e93fa7d010', 'request_id': 'req-7fdfdb45-37da-491c-850a-bf7dad86231e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.325443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02351bab6e5041a7969b50e93fa7d010', 'request_id': 'req-7fdfdb45-37da-491c-850a-bf7dad86231e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.341586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02351bab6e5041a7969b50e93fa7d010', 'request_id': 'req-7fdfdb45-37da-491c-850a-bf7dad86231e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.362572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] The token cache is being invalidate because project 02351bab6e5041a7969b50e93fa7d010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:47.362911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.364842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f7001-5189-44be-b604-6e3e7f8f685c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:47.365396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7fdfdb45-37da-491c-850a-bf7dad86231e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.379874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 661/1322] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:47 2022] DELETE /identity/v3/projects/02351bab6e5041a7969b50e93fa7d010 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:27:47.382864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337499-e2da-4e4c-a411-07f687c6cd09 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.383144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337499-e2da-4e4c-a411-07f687c6cd09 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.383536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337499-e2da-4e4c-a411-07f687c6cd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.428601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c3337499-e2da-4e4c-a411-07f687c6cd09 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:47.431024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c3337499-e2da-4e4c-a411-07f687c6cd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:47.511856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:47.513481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.513990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.514499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] PATH_INFO: `/v3/users/4f9775dfc80e4aaaba521a699eb3f664` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.517784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9775dfc80e4aaaba521a699eb3f664)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:47.522192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:47.562779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 662/1323] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:47.571113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac979ac-34df-4162-9b6b-21b4259a0ac5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.571113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac979ac-34df-4162-9b6b-21b4259a0ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.571443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac979ac-34df-4162-9b6b-21b4259a0ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.608424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5ac979ac-34df-4162-9b6b-21b4259a0ac5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:47.609662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ac979ac-34df-4162-9b6b-21b4259a0ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:47.617599 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9775dfc80e4aaaba521a699eb3f664'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.619338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9775dfc80e4aaaba521a699eb3f664'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.632339 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 663/1324] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:47.636149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac979ac-34df-4162-9b6b-21b4259a0ac5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:47.640847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9775dfc80e4aaaba521a699eb3f664'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.647960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac979ac-34df-4162-9b6b-21b4259a0ac5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:47.653436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:47.654287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.654655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.654936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.655666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:47.656121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:47.658192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:47.662539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9775dfc80e4aaaba521a699eb3f664'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.663420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3df2e489-8e68-41b7-96b4-42bb8c16fbfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9775dfc80e4aaaba521a699eb3f664'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:47.673669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 664/1325] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:27:47.678078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:47.689252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 662/1326] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:47 2022] DELETE /identity/v3/users/4f9775dfc80e4aaaba521a699eb3f664 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:47.691486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad861d-53e7-41fb-b04d-28c9de0c6d47 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.691884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad861d-53e7-41fb-b04d-28c9de0c6d47 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.692215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad861d-53e7-41fb-b04d-28c9de0c6d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.711648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01645b-b339-4fbb-8881-4fc2a251f3aa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:47.722700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8cad861d-53e7-41fb-b04d-28c9de0c6d47 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:47.724012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cad861d-53e7-41fb-b04d-28c9de0c6d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:47.809970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:47.844453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd1b0d-a992-4666-b4be-6ae1f8cb07af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:47.845154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd1b0d-a992-4666-b4be-6ae1f8cb07af admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:47.845433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd1b0d-a992-4666-b4be-6ae1f8cb07af admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:47.847899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 663/1327] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:47.848199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd1b0d-a992-4666-b4be-6ae1f8cb07af admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:47.849183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd1b0d-a992-4666-b4be-6ae1f8cb07af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:47.860131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd1b0d-a992-4666-b4be-6ae1f8cb07af admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:47.888971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad861d-53e7-41fb-b04d-28c9de0c6d47 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:47.891637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cad861d-53e7-41fb-b04d-28c9de0c6d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:47.915380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 665/1328] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:47 2022] POST /identity/v3/projects => generated 363 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:47.916521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd1b0d-a992-4666-b4be-6ae1f8cb07af admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:47.996353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad861d-53e7-41fb-b04d-28c9de0c6d47 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:47.997317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:48.063934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd1b0d-a992-4666-b4be-6ae1f8cb07af admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:48.080951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195bde5-81be-448f-9333-911278771d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:48.081592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195bde5-81be-448f-9333-911278771d81 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:48.081866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195bde5-81be-448f-9333-911278771d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:48.082168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195bde5-81be-448f-9333-911278771d81 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:48.082824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195bde5-81be-448f-9333-911278771d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:48.084787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195bde5-81be-448f-9333-911278771d81 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:48.145592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 666/1329] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:47 2022] POST /identity/v3/users => generated 282 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:48.146285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db346d-bf18-4885-8683-d3d2b540b74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:48.148454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db346d-bf18-4885-8683-d3d2b540b74e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:48.148615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db346d-bf18-4885-8683-d3d2b540b74e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:48.148946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db346d-bf18-4885-8683-d3d2b540b74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:48.152350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195bde5-81be-448f-9333-911278771d81 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:48.176159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195bde5-81be-448f-9333-911278771d81 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:48.177656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db346d-bf18-4885-8683-d3d2b540b74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:48.179283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db346d-bf18-4885-8683-d3d2b540b74e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:48.191051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04993f-8fbe-40b9-8176-db1187cf62f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:48.192391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04993f-8fbe-40b9-8176-db1187cf62f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:48.192705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04993f-8fbe-40b9-8176-db1187cf62f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:48.192981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04993f-8fbe-40b9-8176-db1187cf62f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:48.193685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04993f-8fbe-40b9-8176-db1187cf62f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:48.211043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 664/1330] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:48.215875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04993f-8fbe-40b9-8176-db1187cf62f2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:48.239401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db346d-bf18-4885-8683-d3d2b540b74e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:48.247798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 667/1331] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:48.269072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db346d-bf18-4885-8683-d3d2b540b74e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:48.290109 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69736b4a-89b2-4ab4-befd-8c318cdc070e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:48.290754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69736b4a-89b2-4ab4-befd-8c318cdc070e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:48.291039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69736b4a-89b2-4ab4-befd-8c318cdc070e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:48.294860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69736b4a-89b2-4ab4-befd-8c318cdc070e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:48.295754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69736b4a-89b2-4ab4-befd-8c318cdc070e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:48.300156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69736b4a-89b2-4ab4-befd-8c318cdc070e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:48.334481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 665/1332] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:27:48.339029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04993f-8fbe-40b9-8176-db1187cf62f2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:48.355180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04993f-8fbe-40b9-8176-db1187cf62f2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:48.371844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61443478-c0a1-44c2-ad33-378eb6a56d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:48.372538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61443478-c0a1-44c2-ad33-378eb6a56d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:48.372813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61443478-c0a1-44c2-ad33-378eb6a56d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:48.373089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61443478-c0a1-44c2-ad33-378eb6a56d36 admin admin] PATH_INFO: `/v3/projects/905e6ec27d684627ad4acbe3e1577e50/users/d16ebfdfd9104c2d8bd16bd3c1814066/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:48.418285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61443478-c0a1-44c2-ad33-378eb6a56d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=905e6ec27d684627ad4acbe3e1577e50, user_id=d16ebfdfd9104c2d8bd16bd3c1814066, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:48.424863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61443478-c0a1-44c2-ad33-378eb6a56d36 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:48.464902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 668/1333] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:48 2022] PUT /identity/v3/projects/905e6ec27d684627ad4acbe3e1577e50/users/d16ebfdfd9104c2d8bd16bd3c1814066/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:48.468800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362421d-3731-4e7a-aa6c-94fd774b259c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:48.469090 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362421d-3731-4e7a-aa6c-94fd774b259c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:48.469344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362421d-3731-4e7a-aa6c-94fd774b259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:48.538817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9362421d-3731-4e7a-aa6c-94fd774b259c None None] MFA Rules not processed for user `d16ebfdfd9104c2d8bd16bd3c1814066`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:48.540264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9362421d-3731-4e7a-aa6c-94fd774b259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:48.691441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 666/1334] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:48.696670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61443478-c0a1-44c2-ad33-378eb6a56d36 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:48.802562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61443478-c0a1-44c2-ad33-378eb6a56d36 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:48.810971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79cae-046f-4c33-a170-2d4798cd3a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:48.813361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79cae-046f-4c33-a170-2d4798cd3a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:48.814195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79cae-046f-4c33-a170-2d4798cd3a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:48.814195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79cae-046f-4c33-a170-2d4798cd3a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:48.815110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e79cae-046f-4c33-a170-2d4798cd3a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:48.818313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e79cae-046f-4c33-a170-2d4798cd3a97 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:48.896485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 669/1335] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:48 2022] POST /identity/v3/projects => generated 363 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:48.900702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362421d-3731-4e7a-aa6c-94fd774b259c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:48.916928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362421d-3731-4e7a-aa6c-94fd774b259c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:48.940622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:49.007316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5578b-5bc9-46b5-90f1-1649294481af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:49.008841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5578b-5bc9-46b5-90f1-1649294481af admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.009224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5578b-5bc9-46b5-90f1-1649294481af admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.009673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5578b-5bc9-46b5-90f1-1649294481af admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.011141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5578b-5bc9-46b5-90f1-1649294481af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:49.014254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5578b-5bc9-46b5-90f1-1649294481af admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:49.063668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 667/1336] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:48 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:27:49.068540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79cae-046f-4c33-a170-2d4798cd3a97 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:49.089375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79cae-046f-4c33-a170-2d4798cd3a97 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:49.096534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:49.097261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.098747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.099034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.099956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:49.102319 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:49.145943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 670/1337] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:49.161504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5578b-5bc9-46b5-90f1-1649294481af admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:49.191062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5578b-5bc9-46b5-90f1-1649294481af admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:49.203201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b279f8-152d-4d16-9f8b-bf0a1cd5a02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:49.204990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b279f8-152d-4d16-9f8b-bf0a1cd5a02b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.205442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b279f8-152d-4d16-9f8b-bf0a1cd5a02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.205857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b279f8-152d-4d16-9f8b-bf0a1cd5a02b admin admin] PATH_INFO: `/v3/projects/0542bc5418a8476181a9dec6a4ee33f8/users/43b134c00b164522941ecb903c3930c6/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.216573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:49.220660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:49.238775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b279f8-152d-4d16-9f8b-bf0a1cd5a02b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0542bc5418a8476181a9dec6a4ee33f8, user_id=43b134c00b164522941ecb903c3930c6, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:49.246927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13be14-4d96-4080-a7d5-a048c96b5bd3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:49.247138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b279f8-152d-4d16-9f8b-bf0a1cd5a02b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:49.262536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:49.263595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.264140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.264619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] PATH_INFO: `/v3/projects/66b62e7dcf4a4d4e9ba8f7104e06f2fc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.268520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b62e7dcf4a4d4e9ba8f7104e06f2fc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:49.271049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:49.287928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 668/1338] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:49 2022] PUT /identity/v3/projects/0542bc5418a8476181a9dec6a4ee33f8/users/43b134c00b164522941ecb903c3930c6/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:49.292061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b279f8-152d-4d16-9f8b-bf0a1cd5a02b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:49.397752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b62e7dcf4a4d4e9ba8f7104e06f2fc', 'request_id': 'req-e67222b4-db97-4f3a-9e8d-c1907cb2f988'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:49.399263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:49.410771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b62e7dcf4a4d4e9ba8f7104e06f2fc', 'request_id': 'req-e67222b4-db97-4f3a-9e8d-c1907cb2f988'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:49.431456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b62e7dcf4a4d4e9ba8f7104e06f2fc', 'request_id': 'req-e67222b4-db97-4f3a-9e8d-c1907cb2f988'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:49.458123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] The token cache is being invalidate because project 66b62e7dcf4a4d4e9ba8f7104e06f2fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:49.458463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:49.459433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67222b4-db97-4f3a-9e8d-c1907cb2f988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:49.472153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 671/1339] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:49 2022] DELETE /identity/v3/projects/66b62e7dcf4a4d4e9ba8f7104e06f2fc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:49.473946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefb23c-f7ad-4477-b563-809c8a75f2e6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.474328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefb23c-f7ad-4477-b563-809c8a75f2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.474677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefb23c-f7ad-4477-b563-809c8a75f2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.492590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b279f8-152d-4d16-9f8b-bf0a1cd5a02b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:49.511900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eeefb23c-f7ad-4477-b563-809c8a75f2e6 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:49.518693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eeefb23c-f7ad-4477-b563-809c8a75f2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:49.660229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:49.661718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.662190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.662627 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.663586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:49.665812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:49.706690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 672/1340] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:49.716118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3882-f272-4662-a75c-59dbf32bc02d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.716546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3882-f272-4662-a75c-59dbf32bc02d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.716829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3882-f272-4662-a75c-59dbf32bc02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.721977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 669/1341] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:49.723324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:49.729658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:49.751629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875e97-e2ec-48c9-bacc-8d89864e4318 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:49.763317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7e64d-f019-48e2-bdf4-5ade9641cd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:49.764169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7e64d-f019-48e2-bdf4-5ade9641cd1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.764522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7e64d-f019-48e2-bdf4-5ade9641cd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.764906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7e64d-f019-48e2-bdf4-5ade9641cd1a admin admin] PATH_INFO: `/v3/projects/0542bc5418a8476181a9dec6a4ee33f8/users/43b134c00b164522941ecb903c3930c6/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.767339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7e64d-f019-48e2-bdf4-5ade9641cd1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0542bc5418a8476181a9dec6a4ee33f8, user_id=43b134c00b164522941ecb903c3930c6, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:49.770528 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7e64d-f019-48e2-bdf4-5ade9641cd1a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:49.775575 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-503a3882-f272-4662-a75c-59dbf32bc02d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:49.777087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-503a3882-f272-4662-a75c-59dbf32bc02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:49.797376 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 670/1342] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:49 2022] PUT /identity/v3/projects/0542bc5418a8476181a9dec6a4ee33f8/users/43b134c00b164522941ecb903c3930c6/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:49.819918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2761e3-3dbc-4939-a69a-5ac7ddc33fb1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.820269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2761e3-3dbc-4939-a69a-5ac7ddc33fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.820649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2761e3-3dbc-4939-a69a-5ac7ddc33fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.867365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0c2761e3-3dbc-4939-a69a-5ac7ddc33fb1 None None] MFA Rules not processed for user `43b134c00b164522941ecb903c3930c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:49.871537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c2761e3-3dbc-4939-a69a-5ac7ddc33fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:49.883929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:49.944153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 673/1343] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:49.949331 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3882-f272-4662-a75c-59dbf32bc02d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:49.962278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3882-f272-4662-a75c-59dbf32bc02d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:49.979363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:49.980599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:49.980893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:49.981172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:49.981839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:49.982170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:49.983953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.020461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 674/1344] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:27:50.024525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.051966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744bf6c-aaf9-496f-99d4-b59c9ccb4029 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.061594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f0ccf-cd0d-4c99-b97e-11dcbf743b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.062278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0ccf-cd0d-4c99-b97e-11dcbf743b09 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.062576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0ccf-cd0d-4c99-b97e-11dcbf743b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.062858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0ccf-cd0d-4c99-b97e-11dcbf743b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.063671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f0ccf-cd0d-4c99-b97e-11dcbf743b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.065654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f0ccf-cd0d-4c99-b97e-11dcbf743b09 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.108015 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 671/1345] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.116968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2761e3-3dbc-4939-a69a-5ac7ddc33fb1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.119307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c2761e3-3dbc-4939-a69a-5ac7ddc33fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.136125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 675/1346] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:50.143133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f0ccf-cd0d-4c99-b97e-11dcbf743b09 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.244975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:50.246565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2761e3-3dbc-4939-a69a-5ac7ddc33fb1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.279648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f0ccf-cd0d-4c99-b97e-11dcbf743b09 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.292107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c473513-d85a-4eeb-b33b-eb44a52d11bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.292749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c473513-d85a-4eeb-b33b-eb44a52d11bd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.293033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c473513-d85a-4eeb-b33b-eb44a52d11bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.293307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c473513-d85a-4eeb-b33b-eb44a52d11bd admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.293954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c473513-d85a-4eeb-b33b-eb44a52d11bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.295995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c473513-d85a-4eeb-b33b-eb44a52d11bd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.346141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 676/1347] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:27:50.347682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c473513-d85a-4eeb-b33b-eb44a52d11bd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.362662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801f1b3-608b-4bda-a09f-2fdcdc2a2e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.363305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f1b3-608b-4bda-a09f-2fdcdc2a2e6d service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.363587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f1b3-608b-4bda-a09f-2fdcdc2a2e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.363860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f1b3-608b-4bda-a09f-2fdcdc2a2e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.382754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c473513-d85a-4eeb-b33b-eb44a52d11bd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.413389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0799-fc95-4dc2-82d4-2b2090143a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.414028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0799-fc95-4dc2-82d4-2b2090143a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.414343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0799-fc95-4dc2-82d4-2b2090143a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.414617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0799-fc95-4dc2-82d4-2b2090143a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.415276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf0799-fc95-4dc2-82d4-2b2090143a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.421087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf0799-fc95-4dc2-82d4-2b2090143a8a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.457801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 677/1348] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:27:50.461594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0799-fc95-4dc2-82d4-2b2090143a8a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.491192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0799-fc95-4dc2-82d4-2b2090143a8a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.497529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098dae9-2676-4668-81cc-2689765907da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.498586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098dae9-2676-4668-81cc-2689765907da admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.499299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098dae9-2676-4668-81cc-2689765907da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.499795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098dae9-2676-4668-81cc-2689765907da admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.500636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098dae9-2676-4668-81cc-2689765907da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.502613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098dae9-2676-4668-81cc-2689765907da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.506468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801f1b3-608b-4bda-a09f-2fdcdc2a2e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.507944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801f1b3-608b-4bda-a09f-2fdcdc2a2e6d service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.517978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 678/1349] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:27:50.523432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098dae9-2676-4668-81cc-2689765907da admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.544727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098dae9-2676-4668-81cc-2689765907da admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.554052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.555939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.556481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.556844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] PATH_INFO: `/v3/projects/2526a56fc3ec4267b657384213c6ffee/users/a021ef8ff93f4572b933c07fcd359184/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.568761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 672/1350] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:50.604781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2526a56fc3ec4267b657384213c6ffee, user_id=a021ef8ff93f4572b933c07fcd359184, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.613214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.653170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 679/1351] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:50 2022] PUT /identity/v3/projects/2526a56fc3ec4267b657384213c6ffee/users/a021ef8ff93f4572b933c07fcd359184/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:50.656068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ba82-1062-476d-8b1a-78eaf7c63aec None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.656068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ba82-1062-476d-8b1a-78eaf7c63aec None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.656068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ba82-1062-476d-8b1a-78eaf7c63aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.674191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:50.687554 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.691126 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.716893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5c60ba82-1062-476d-8b1a-78eaf7c63aec None None] MFA Rules not processed for user `a021ef8ff93f4572b933c07fcd359184`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.722800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5c60ba82-1062-476d-8b1a-78eaf7c63aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.879710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 673/1352] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.887271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60ba82-1062-476d-8b1a-78eaf7c63aec None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.903279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dec2fd-cce2-4c94-a1e2-c367af4f3970 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.917110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.919512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.920041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.920533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] PATH_INFO: `/v3/projects/7150594e44c241a6a4a6110b0a676cc0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.923204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60ba82-1062-476d-8b1a-78eaf7c63aec None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.924441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7150594e44c241a6a4a6110b0a676cc0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.928083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.938339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ebe99-0b56-47c8-9e5b-ef767858e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.938869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe99-0b56-47c8-9e5b-ef767858e92e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.939261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe99-0b56-47c8-9e5b-ef767858e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.939792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe99-0b56-47c8-9e5b-ef767858e92e admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.942002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ebe99-0b56-47c8-9e5b-ef767858e92e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.947422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ebe99-0b56-47c8-9e5b-ef767858e92e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.053552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7150594e44c241a6a4a6110b0a676cc0', 'request_id': 'req-d9c50693-317d-4e68-b8c1-5535cc94ff5a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.085084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 674/1353] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.086582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ebe99-0b56-47c8-9e5b-ef767858e92e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.113559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7150594e44c241a6a4a6110b0a676cc0', 'request_id': 'req-d9c50693-317d-4e68-b8c1-5535cc94ff5a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.140497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7150594e44c241a6a4a6110b0a676cc0', 'request_id': 'req-d9c50693-317d-4e68-b8c1-5535cc94ff5a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.163543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] The token cache is being invalidate because project 7150594e44c241a6a4a6110b0a676cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:51.164685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.173209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9c50693-317d-4e68-b8c1-5535cc94ff5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.188043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 680/1354] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:50 2022] DELETE /identity/v3/projects/7150594e44c241a6a4a6110b0a676cc0 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:51.189306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a680-8129-47b2-b458-ce36c03354ca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.189599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a680-8129-47b2-b458-ce36c03354ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.189874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a680-8129-47b2-b458-ce36c03354ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.243245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1553a680-8129-47b2-b458-ce36c03354ca None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:51.244491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1553a680-8129-47b2-b458-ce36c03354ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:51.304383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:51.315483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ebe99-0b56-47c8-9e5b-ef767858e92e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.322010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.322715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.323047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.323384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.324166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.328045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.369892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 675/1355] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/users => generated 299 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.374204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.377982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:51.457778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 681/1356] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:51.461162 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0a7a6-7ab4-41d0-b1e1-ea498b2ca39b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.466661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255964a9-e6da-47dc-a9e6-a50893e7e77b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.466967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255964a9-e6da-47dc-a9e6-a50893e7e77b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.467251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255964a9-e6da-47dc-a9e6-a50893e7e77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.490746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.495539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.498829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.505378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.507062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.508857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.520372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-255964a9-e6da-47dc-a9e6-a50893e7e77b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:51.521650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-255964a9-e6da-47dc-a9e6-a50893e7e77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:51.539630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 676/1357] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.540531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.546824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:51.562694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 682/1358] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:51.563659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255964a9-e6da-47dc-a9e6-a50893e7e77b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.567788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f1ddb-f34d-420b-a666-c0945e85086e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.598859 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.600122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.600424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.600699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] PATH_INFO: `/v3/users/d16ebfdfd9104c2d8bd16bd3c1814066` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.604630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d16ebfdfd9104c2d8bd16bd3c1814066)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.608520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.613280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255964a9-e6da-47dc-a9e6-a50893e7e77b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.624141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd24b55-e16a-4bc7-8f90-927b0cc80c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.624850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd24b55-e16a-4bc7-8f90-927b0cc80c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.625128 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd24b55-e16a-4bc7-8f90-927b0cc80c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.625405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd24b55-e16a-4bc7-8f90-927b0cc80c2c admin admin] PATH_INFO: `/v3/projects/0f51f3eb09864c5cb045f7c198e2f2d2/users/9e3648eb27f947e3ad57498efe1bbecf/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.680665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd24b55-e16a-4bc7-8f90-927b0cc80c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f51f3eb09864c5cb045f7c198e2f2d2, user_id=9e3648eb27f947e3ad57498efe1bbecf, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.683503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd24b55-e16a-4bc7-8f90-927b0cc80c2c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.726630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 683/1359] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:51 2022] PUT /identity/v3/projects/0f51f3eb09864c5cb045f7c198e2f2d2/users/9e3648eb27f947e3ad57498efe1bbecf/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:51.727678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd24b55-e16a-4bc7-8f90-927b0cc80c2c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.738204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd16ebfdfd9104c2d8bd16bd3c1814066'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.739614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd16ebfdfd9104c2d8bd16bd3c1814066'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.747156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:51.787616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd16ebfdfd9104c2d8bd16bd3c1814066'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.816442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd16ebfdfd9104c2d8bd16bd3c1814066'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.817557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd16ebfdfd9104c2d8bd16bd3c1814066'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:51.863279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 677/1360] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:51 2022] DELETE /identity/v3/users/d16ebfdfd9104c2d8bd16bd3c1814066 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:51.866203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.869334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:51.889356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd24b55-e16a-4bc7-8f90-927b0cc80c2c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.051204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae3ded-73f1-4ef4-ba39-880e56d275f0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.059466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b874c-c7f6-4f69-a898-8c6c2eaf82e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.066341 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b874c-c7f6-4f69-a898-8c6c2eaf82e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.066715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b874c-c7f6-4f69-a898-8c6c2eaf82e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.066996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b874c-c7f6-4f69-a898-8c6c2eaf82e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.067734 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b874c-c7f6-4f69-a898-8c6c2eaf82e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.069519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b874c-c7f6-4f69-a898-8c6c2eaf82e1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.080288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.081145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.081563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.083532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.084339 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:52.087374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.090135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.102835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 678/1361] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.102835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba491f6-74f8-410e-a311-6e85bddc3a37 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.105449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba491f6-74f8-410e-a311-6e85bddc3a37 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.105844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba491f6-74f8-410e-a311-6e85bddc3a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.140693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 684/1362] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:52.145201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.160856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0ba491f6-74f8-410e-a311-6e85bddc3a37 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.162431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0ba491f6-74f8-410e-a311-6e85bddc3a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.178026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275e56-184c-467c-ac36-e4d714687d2c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.179136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 679/1363] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.180132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba491f6-74f8-410e-a311-6e85bddc3a37 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.186616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0ba491f6-74f8-410e-a311-6e85bddc3a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.203543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fbb9c-b4cf-4e13-a18b-93a145a055a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.205836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fbb9c-b4cf-4e13-a18b-93a145a055a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.206281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fbb9c-b4cf-4e13-a18b-93a145a055a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.207500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fbb9c-b4cf-4e13-a18b-93a145a055a2 admin admin] PATH_INFO: `/v3/projects/0f51f3eb09864c5cb045f7c198e2f2d2/users/9e3648eb27f947e3ad57498efe1bbecf/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.214537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba491f6-74f8-410e-a311-6e85bddc3a37 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.217858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fbb9c-b4cf-4e13-a18b-93a145a055a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f51f3eb09864c5cb045f7c198e2f2d2, user_id=9e3648eb27f947e3ad57498efe1bbecf, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.221329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fbb9c-b4cf-4e13-a18b-93a145a055a2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.225260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f07328-ba2a-4bc3-804d-d03c1ca2cde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.226185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f07328-ba2a-4bc3-804d-d03c1ca2cde0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.226633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f07328-ba2a-4bc3-804d-d03c1ca2cde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.227069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f07328-ba2a-4bc3-804d-d03c1ca2cde0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.227903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f07328-ba2a-4bc3-804d-d03c1ca2cde0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.230044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f07328-ba2a-4bc3-804d-d03c1ca2cde0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.277343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 685/1364] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:52 2022] PUT /identity/v3/projects/0f51f3eb09864c5cb045f7c198e2f2d2/users/9e3648eb27f947e3ad57498efe1bbecf/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:52.280086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d34f16-63ce-4e14-a804-785934797bca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.284638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d34f16-63ce-4e14-a804-785934797bca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.285042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d34f16-63ce-4e14-a804-785934797bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.320679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 680/1365] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/projects => generated 403 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:52.326300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f07328-ba2a-4bc3-804d-d03c1ca2cde0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.382525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-48d34f16-63ce-4e14-a804-785934797bca None None] MFA Rules not processed for user `9e3648eb27f947e3ad57498efe1bbecf`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.388810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-48d34f16-63ce-4e14-a804-785934797bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.402495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:52.539863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f07328-ba2a-4bc3-804d-d03c1ca2cde0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.566637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.572428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.572428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.572711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.575586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.603199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.717767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 681/1366] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/users => generated 302 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:52.723175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.731401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.738317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 686/1367] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.742971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d34f16-63ce-4e14-a804-785934797bca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.789319 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d34f16-63ce-4e14-a804-785934797bca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.808457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.809103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.815894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.816351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.817491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.825061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.854541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:52.873525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 687/1368] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.873525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.883797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.984379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3352f-eec2-4dd6-8ca9-cec791703266 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.997459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5819d90-245f-468d-a448-eee87c336594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.999866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819d90-245f-468d-a448-eee87c336594 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.000168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819d90-245f-468d-a448-eee87c336594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.000446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819d90-245f-468d-a448-eee87c336594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.038059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5819d90-245f-468d-a448-eee87c336594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.040881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5819d90-245f-468d-a448-eee87c336594 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.057263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c061e3-2c59-4954-a3ed-efeeb414c740 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.062976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3145c-a1ac-46b4-ad45-7da5aaa9fe4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.063753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3145c-a1ac-46b4-ad45-7da5aaa9fe4a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.064129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3145c-a1ac-46b4-ad45-7da5aaa9fe4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.064582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3145c-a1ac-46b4-ad45-7da5aaa9fe4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.071449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 682/1369] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:53.072690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5819d90-245f-468d-a448-eee87c336594 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.093283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3145c-a1ac-46b4-ad45-7da5aaa9fe4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.097213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3145c-a1ac-46b4-ad45-7da5aaa9fe4a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.099026 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5819d90-245f-468d-a448-eee87c336594 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.106802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cc927-4198-4584-be5c-d32e312b3415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.107460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cc927-4198-4584-be5c-d32e312b3415 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.107739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cc927-4198-4584-be5c-d32e312b3415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.108006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cc927-4198-4584-be5c-d32e312b3415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.108646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cc927-4198-4584-be5c-d32e312b3415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.111257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 688/1370] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:53.112323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cc927-4198-4584-be5c-d32e312b3415 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.138299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 683/1371] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:53.143739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3145c-a1ac-46b4-ad45-7da5aaa9fe4a service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.167817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3145c-a1ac-46b4-ad45-7da5aaa9fe4a service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.180588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db606a26-6fe7-4d46-b133-bee90b8443d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.181691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606a26-6fe7-4d46-b133-bee90b8443d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.181980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606a26-6fe7-4d46-b133-bee90b8443d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.183099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606a26-6fe7-4d46-b133-bee90b8443d4 admin admin] PATH_INFO: `/v3/projects/53ab18099672426e826cab1501379935/users/8db17a890cce4b64914e5c883b3d022d/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.242602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db606a26-6fe7-4d46-b133-bee90b8443d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ab18099672426e826cab1501379935, user_id=8db17a890cce4b64914e5c883b3d022d, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.246134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db606a26-6fe7-4d46-b133-bee90b8443d4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.286040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 689/1372] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:53 2022] PUT /identity/v3/projects/53ab18099672426e826cab1501379935/users/8db17a890cce4b64914e5c883b3d022d/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:53.292360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55118-cf4e-49cd-b790-9d9f3bfc6866 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.293695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55118-cf4e-49cd-b790-9d9f3bfc6866 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.294277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55118-cf4e-49cd-b790-9d9f3bfc6866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.346159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0c6c2-be26-46b8-9bc8-dfdd456ad603 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.347275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0c6c2-be26-46b8-9bc8-dfdd456ad603 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.347577 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0c6c2-be26-46b8-9bc8-dfdd456ad603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.349831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cdd55118-cf4e-49cd-b790-9d9f3bfc6866 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:53.352058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cdd55118-cf4e-49cd-b790-9d9f3bfc6866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:53.431436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-57e0c6c2-be26-46b8-9bc8-dfdd456ad603 None None] MFA Rules not processed for user `8f151dc283e8498f961b49320130f770`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:53.434652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-57e0c6c2-be26-46b8-9bc8-dfdd456ad603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:53.535287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 684/1373] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:53.558565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55118-cf4e-49cd-b790-9d9f3bfc6866 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.560119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cdd55118-cf4e-49cd-b790-9d9f3bfc6866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:53.636163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:53.665174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 690/1374] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:53.672007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0c6c2-be26-46b8-9bc8-dfdd456ad603 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.673802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-57e0c6c2-be26-46b8-9bc8-dfdd456ad603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:53.763591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55118-cf4e-49cd-b790-9d9f3bfc6866 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.775818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a79eef-6a97-48f6-82c8-7d99a10cb619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.776553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79eef-6a97-48f6-82c8-7d99a10cb619 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.782592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79eef-6a97-48f6-82c8-7d99a10cb619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.783047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79eef-6a97-48f6-82c8-7d99a10cb619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.807142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a79eef-6a97-48f6-82c8-7d99a10cb619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.808552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a79eef-6a97-48f6-82c8-7d99a10cb619 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.836486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 685/1375] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:53.913750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0c6c2-be26-46b8-9bc8-dfdd456ad603 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.938863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.939498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.939770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.940039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.990466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.999359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:54.023905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 691/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:54.484561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a79eef-6a97-48f6-82c8-7d99a10cb619 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:54.495536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a79eef-6a97-48f6-82c8-7d99a10cb619 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:54.500603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:54.501362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.501637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.501970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] PATH_INFO: `/v3/users/a021ef8ff93f4572b933c07fcd359184` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.503249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a021ef8ff93f4572b933c07fcd359184)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:54.505185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:54.568910 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:54.573759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:54.606335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30267c6a-33ec-4d3a-af87-5e1c7878bff7 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:54.616293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a021ef8ff93f4572b933c07fcd359184'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.617993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:54.618447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a021ef8ff93f4572b933c07fcd359184'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.619903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.623238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.624032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] PATH_INFO: `/v3/projects/905e6ec27d684627ad4acbe3e1577e50` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.627414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] RBAC: Authorizing `identity:delete_project(project_id=905e6ec27d684627ad4acbe3e1577e50)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:54.633365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:54.634345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:54.651840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a021ef8ff93f4572b933c07fcd359184'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.661982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a021ef8ff93f4572b933c07fcd359184'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.663065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed8a4fd2-6662-4f5d-b382-13755e291e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a021ef8ff93f4572b933c07fcd359184'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.694119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 686/1377] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:54 2022] DELETE /identity/v3/users/a021ef8ff93f4572b933c07fcd359184 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:54.696716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaf36c-0f80-4274-8a70-9f1dfa6c3cd9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.697231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaf36c-0f80-4274-8a70-9f1dfa6c3cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.697653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaf36c-0f80-4274-8a70-9f1dfa6c3cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.786559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '905e6ec27d684627ad4acbe3e1577e50', 'request_id': 'req-f11c8712-dd4d-453c-b914-a6223681291b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.801349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8ebaf36c-0f80-4274-8a70-9f1dfa6c3cd9 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:54.807519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8ebaf36c-0f80-4274-8a70-9f1dfa6c3cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:54.815984 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '905e6ec27d684627ad4acbe3e1577e50', 'request_id': 'req-f11c8712-dd4d-453c-b914-a6223681291b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.837052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '905e6ec27d684627ad4acbe3e1577e50', 'request_id': 'req-f11c8712-dd4d-453c-b914-a6223681291b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.854561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] The token cache is being invalidate because project 905e6ec27d684627ad4acbe3e1577e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:54.854856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.855589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.861825 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 692/1378] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:54 2022] DELETE /identity/v3/projects/905e6ec27d684627ad4acbe3e1577e50 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:27:54.863187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:54.865471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.011396 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 687/1379] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:55.029023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaf36c-0f80-4274-8a70-9f1dfa6c3cd9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:55.030643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8ebaf36c-0f80-4274-8a70-9f1dfa6c3cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.074397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c8712-dd4d-453c-b914-a6223681291b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:55.088018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:55.088671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.088991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.089328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] PATH_INFO: `/v3/users/43b134c00b164522941ecb903c3930c6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.091063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b134c00b164522941ecb903c3930c6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:55.096609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:55.112943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:55.201249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaf36c-0f80-4274-8a70-9f1dfa6c3cd9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:55.210842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea25426-6549-49df-b74c-e190b7909c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:55.211265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea25426-6549-49df-b74c-e190b7909c97 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.211544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea25426-6549-49df-b74c-e190b7909c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.211898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea25426-6549-49df-b74c-e190b7909c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.230110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea25426-6549-49df-b74c-e190b7909c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:55.232048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea25426-6549-49df-b74c-e190b7909c97 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:55.236356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b134c00b164522941ecb903c3930c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:55.237189 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b134c00b164522941ecb903c3930c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:55.256815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b134c00b164522941ecb903c3930c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:55.266548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 688/1380] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:55.284815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b134c00b164522941ecb903c3930c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:55.286795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b134c00b164522941ecb903c3930c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:55.319444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 693/1381] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:54 2022] DELETE /identity/v3/users/43b134c00b164522941ecb903c3930c6 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:56.370471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c6c1-11c3-4827-a3d3-2c9d76d641d4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.371042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c6c1-11c3-4827-a3d3-2c9d76d641d4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.371042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c6c1-11c3-4827-a3d3-2c9d76d641d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.500491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4088c6c1-11c3-4827-a3d3-2c9d76d641d4 None None] MFA Rules not processed for user `0e8717d368d44102bea0eb82fbee8a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:56.501924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4088c6c1-11c3-4827-a3d3-2c9d76d641d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:56.606689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:56.612579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:56.704373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 689/1382] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:56.717046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088c6c1-11c3-4827-a3d3-2c9d76d641d4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:56.718868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4088c6c1-11c3-4827-a3d3-2c9d76d641d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:56.733479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:56.796541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736551c6-1109-4b50-99da-ed9d6e38086f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:56.812958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:56.813758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.814042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.814488 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] PATH_INFO: `/v3/users/af51aa4023f949338cb0033af92a7c2b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.817815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af51aa4023f949338cb0033af92a7c2b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:56.820445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:57.046721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088c6c1-11c3-4827-a3d3-2c9d76d641d4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:57.059598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af51aa4023f949338cb0033af92a7c2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.060323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af51aa4023f949338cb0033af92a7c2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.073703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10495de-58fc-457b-b3b4-348f0f183f40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:57.076219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10495de-58fc-457b-b3b4-348f0f183f40 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:57.077994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10495de-58fc-457b-b3b4-348f0f183f40 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:57.078330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10495de-58fc-457b-b3b4-348f0f183f40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:57.087609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a10495de-58fc-457b-b3b4-348f0f183f40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.111986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af51aa4023f949338cb0033af92a7c2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.122160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af51aa4023f949338cb0033af92a7c2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.124009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af51aa4023f949338cb0033af92a7c2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.136723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10495de-58fc-457b-b3b4-348f0f183f40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:57.141679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10495de-58fc-457b-b3b4-348f0f183f40 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:57.175842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 694/1383] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:56 2022] DELETE /identity/v3/users/af51aa4023f949338cb0033af92a7c2b => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:57.177147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:57.180362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.191105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:57.195250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 690/1384] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:57.200079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0b620-796c-47f5-9295-202a40a8a271 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:57.200079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0b620-796c-47f5-9295-202a40a8a271 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:57.200079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0b620-796c-47f5-9295-202a40a8a271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:57.256187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e0a0b620-796c-47f5-9295-202a40a8a271 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:57.258244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0a0b620-796c-47f5-9295-202a40a8a271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.388740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f3e0e-543c-45ae-9e22-6a98dad4d3c6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:57.410496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:57.411158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:57.411430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:57.411702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] PATH_INFO: `/v3/projects/2526a56fc3ec4267b657384213c6ffee` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:57.421612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2526a56fc3ec4267b657384213c6ffee)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:57.434763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:57.475458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 691/1385] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:57.568549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0b620-796c-47f5-9295-202a40a8a271 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:57.570523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0a0b620-796c-47f5-9295-202a40a8a271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.575011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2526a56fc3ec4267b657384213c6ffee', 'request_id': 'req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.594415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2526a56fc3ec4267b657384213c6ffee', 'request_id': 'req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.614190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2526a56fc3ec4267b657384213c6ffee', 'request_id': 'req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.631081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] The token cache is being invalidate because project 2526a56fc3ec4267b657384213c6ffee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:57.631486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.633882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.638332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 695/1386] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:57 2022] DELETE /identity/v3/projects/2526a56fc3ec4267b657384213c6ffee => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:57.642396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:57.645217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.729966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0b620-796c-47f5-9295-202a40a8a271 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:57.776871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb82fc-736b-4a55-9d8a-d34eae7cc4a2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:57.788251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:57.788974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:57.790610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:57.790910 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] PATH_INFO: `/v3/users/9e3648eb27f947e3ad57498efe1bbecf` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:57.794615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3648eb27f947e3ad57498efe1bbecf)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:57.799531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:57.838542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:57.863412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:57.864111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:57.864569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:57.864687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:57.868854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.888504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:57.910690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:57.912247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:57.918034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3648eb27f947e3ad57498efe1bbecf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.922200 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3648eb27f947e3ad57498efe1bbecf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.970201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 692/1387] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:57.979300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3648eb27f947e3ad57498efe1bbecf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.000447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3648eb27f947e3ad57498efe1bbecf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.002345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8cc36b41-6aa8-4015-a7b0-953cb45b91fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3648eb27f947e3ad57498efe1bbecf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.026384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 696/1388] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:57 2022] DELETE /identity/v3/users/9e3648eb27f947e3ad57498efe1bbecf => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:27:58.150004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:58.151545 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:58.172411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c495760-db93-460a-9b14-20f2c6dd7b85 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:58.296563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:58.297222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:58.297494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:58.297806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] PATH_INFO: `/v3/projects/0542bc5418a8476181a9dec6a4ee33f8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:58.300193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0542bc5418a8476181a9dec6a4ee33f8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:58.301605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:58.301605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Nov 02 07:27:58.301605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: Nov 02 07:27:58.306350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:58.466010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0542bc5418a8476181a9dec6a4ee33f8', 'request_id': 'req-2406acbc-13d8-416d-9b5b-ba93afa5bb72'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.491851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0542bc5418a8476181a9dec6a4ee33f8', 'request_id': 'req-2406acbc-13d8-416d-9b5b-ba93afa5bb72'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.523830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0542bc5418a8476181a9dec6a4ee33f8', 'request_id': 'req-2406acbc-13d8-416d-9b5b-ba93afa5bb72'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.581172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] The token cache is being invalidate because project 0542bc5418a8476181a9dec6a4ee33f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:58.581172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.581172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2406acbc-13d8-416d-9b5b-ba93afa5bb72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.581172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 693/1389] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:58 2022] DELETE /identity/v3/projects/0542bc5418a8476181a9dec6a4ee33f8 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:58.581556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64497deb-7943-44e6-9847-258e9f87c5ae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:58.581556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64497deb-7943-44e6-9847-258e9f87c5ae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:58.581556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64497deb-7943-44e6-9847-258e9f87c5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:58.632642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-64497deb-7943-44e6-9847-258e9f87c5ae None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:58.634065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-64497deb-7943-44e6-9847-258e9f87c5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:58.797493 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 697/1390] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:58.806486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefae18-cafd-4717-a4bc-b0a55914e342 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:58.806839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefae18-cafd-4717-a4bc-b0a55914e342 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:58.807150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefae18-cafd-4717-a4bc-b0a55914e342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:58.855968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2eefae18-cafd-4717-a4bc-b0a55914e342 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:58.861684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2eefae18-cafd-4717-a4bc-b0a55914e342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:58.884904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 694/1391] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:58.888978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64497deb-7943-44e6-9847-258e9f87c5ae None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:58.907379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64497deb-7943-44e6-9847-258e9f87c5ae None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:58.922478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31461729-13a1-4ff2-ae92-b2eb0a6e7fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:58.924115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31461729-13a1-4ff2-ae92-b2eb0a6e7fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:58.924543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31461729-13a1-4ff2-ae92-b2eb0a6e7fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:58.924969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31461729-13a1-4ff2-ae92-b2eb0a6e7fe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:58.926915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31461729-13a1-4ff2-ae92-b2eb0a6e7fe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:58.927394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31461729-13a1-4ff2-ae92-b2eb0a6e7fe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:58.945968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31461729-13a1-4ff2-ae92-b2eb0a6e7fe2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:58.989329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 698/1392] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:58.991155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefae18-cafd-4717-a4bc-b0a55914e342 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.009494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145a59d-1193-4cf4-86a5-697052baa799 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.011067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145a59d-1193-4cf4-86a5-697052baa799 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.011354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145a59d-1193-4cf4-86a5-697052baa799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.012014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefae18-cafd-4717-a4bc-b0a55914e342 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.036869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f16bd-3a9a-4a95-b14b-4a49cc2be165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:59.039372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f16bd-3a9a-4a95-b14b-4a49cc2be165 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.039988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f16bd-3a9a-4a95-b14b-4a49cc2be165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.040845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f16bd-3a9a-4a95-b14b-4a49cc2be165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.044267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f16bd-3a9a-4a95-b14b-4a49cc2be165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:59.051603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f16bd-3a9a-4a95-b14b-4a49cc2be165 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:59.110385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2145a59d-1193-4cf4-86a5-697052baa799 None None] MFA Rules not processed for user `006ae080f2884719bd4ddabe815d0d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:59.111694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2145a59d-1193-4cf4-86a5-697052baa799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:59.154504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 695/1393] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:58 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:59.158595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f16bd-3a9a-4a95-b14b-4a49cc2be165 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.294075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:59.309070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 699/1394] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:59.312954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e323c-32bf-4b34-9dad-827356ad7854 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.313542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e323c-32bf-4b34-9dad-827356ad7854 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.315187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e323c-32bf-4b34-9dad-827356ad7854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.339814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f16bd-3a9a-4a95-b14b-4a49cc2be165 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.358052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:59.358758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.359166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.359444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.362781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:59.367515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:59.399993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cc9e323c-32bf-4b34-9dad-827356ad7854 None None] MFA Rules not processed for user `8db17a890cce4b64914e5c883b3d022d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:59.403951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc9e323c-32bf-4b34-9dad-827356ad7854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:59.419750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 696/1395] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:59 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:59.420645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.428420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:59.439766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:59.569703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 700/1396] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:27:59 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:59.572086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e323c-32bf-4b34-9dad-827356ad7854 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.611729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e323c-32bf-4b34-9dad-827356ad7854 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.621543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:59.622291 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.623223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.623503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.624170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:59.624641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9a679-ea9a-4e74-83bd-dff7f65388d1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.628850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:59.642351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:59.643085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.643252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.644073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.701548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 701/1397] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:59.704956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.717843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:59.758686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0305d-80e6-4110-ba41-bcd4bd804b2d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.776529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:59.777328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.777670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.778003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] PATH_INFO: `/v3/projects/c1d95dd88d4c44e1a251604ef3a493ba` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.783031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d95dd88d4c44e1a251604ef3a493ba)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:59.786363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:59.823550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:59.825232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:59.838352 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 697/1398] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:59.839216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.855290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:59.889191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a0a12-19fe-40f9-bd1d-e297d72f180b service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.920787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d95dd88d4c44e1a251604ef3a493ba', 'request_id': 'req-cd628209-1371-4f22-aedd-fabc8d652dd4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:59.963179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d95dd88d4c44e1a251604ef3a493ba', 'request_id': 'req-cd628209-1371-4f22-aedd-fabc8d652dd4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:59.998660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d95dd88d4c44e1a251604ef3a493ba', 'request_id': 'req-cd628209-1371-4f22-aedd-fabc8d652dd4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:00.022324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] The token cache is being invalidate because project c1d95dd88d4c44e1a251604ef3a493ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:00.024019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:00.025776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:00.037353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6f6ca-0515-4313-ad31-727871b1bee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.037952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6f6ca-0515-4313-ad31-727871b1bee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.038256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 702/1399] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:59 2022] DELETE /identity/v3/projects/c1d95dd88d4c44e1a251604ef3a493ba => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:00.038372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6f6ca-0515-4313-ad31-727871b1bee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.038531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6f6ca-0515-4313-ad31-727871b1bee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.039176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6f6ca-0515-4313-ad31-727871b1bee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.039749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.041085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6f6ca-0515-4313-ad31-727871b1bee3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.046180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.074527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd628209-1371-4f22-aedd-fabc8d652dd4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.084618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623730c3-8efc-46e0-8bad-1b483e427b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.085320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623730c3-8efc-46e0-8bad-1b483e427b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.085710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623730c3-8efc-46e0-8bad-1b483e427b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.086660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623730c3-8efc-46e0-8bad-1b483e427b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.087473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623730c3-8efc-46e0-8bad-1b483e427b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.089838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623730c3-8efc-46e0-8bad-1b483e427b08 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.111463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:00.118971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 703/1400] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:00.121791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80f08a-6bc8-4a42-a83c-8ab6cd2ed777 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.122332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80f08a-6bc8-4a42-a83c-8ab6cd2ed777 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.127669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80f08a-6bc8-4a42-a83c-8ab6cd2ed777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.130935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 698/1401] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:59 2022] POST /identity/v3/projects => generated 405 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:00.132132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6f6ca-0515-4313-ad31-727871b1bee3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.171087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9b80f08a-6bc8-4a42-a83c-8ab6cd2ed777 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:00.172495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9b80f08a-6bc8-4a42-a83c-8ab6cd2ed777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.278541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6f6ca-0515-4313-ad31-727871b1bee3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.302452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5104482-52e9-4a7f-8cfe-855fb9ab8ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.304025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104482-52e9-4a7f-8cfe-855fb9ab8ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.304310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104482-52e9-4a7f-8cfe-855fb9ab8ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.304948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104482-52e9-4a7f-8cfe-855fb9ab8ad3 admin admin] PATH_INFO: `/v3/projects/74520c9f0a9d48d28ca1b0a35a09bbff/users/a7917d6400b44916b0e0d9cc21d39d72/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.337765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 704/1402] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:00.346946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80f08a-6bc8-4a42-a83c-8ab6cd2ed777 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.347304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5104482-52e9-4a7f-8cfe-855fb9ab8ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74520c9f0a9d48d28ca1b0a35a09bbff, user_id=a7917d6400b44916b0e0d9cc21d39d72, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.350422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5104482-52e9-4a7f-8cfe-855fb9ab8ad3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.351461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9b80f08a-6bc8-4a42-a83c-8ab6cd2ed777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.382370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 699/1403] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:00 2022] PUT /identity/v3/projects/74520c9f0a9d48d28ca1b0a35a09bbff/users/a7917d6400b44916b0e0d9cc21d39d72/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:00.384507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b172-1a3b-4597-8c83-31c90040d5b0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.384863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b172-1a3b-4597-8c83-31c90040d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.385155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b172-1a3b-4597-8c83-31c90040d5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.385968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80f08a-6bc8-4a42-a83c-8ab6cd2ed777 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.425054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:00.431649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f672b172-1a3b-4597-8c83-31c90040d5b0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:00.434547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f672b172-1a3b-4597-8c83-31c90040d5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.525052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4b045-1309-4f63-8c1e-3b6effa71076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.525682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b045-1309-4f63-8c1e-3b6effa71076 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.526014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b045-1309-4f63-8c1e-3b6effa71076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.526347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b045-1309-4f63-8c1e-3b6effa71076 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.527275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4b045-1309-4f63-8c1e-3b6effa71076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.528942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4b045-1309-4f63-8c1e-3b6effa71076 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.583305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 700/1404] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:00.584233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672b172-1a3b-4597-8c83-31c90040d5b0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.586896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f672b172-1a3b-4597-8c83-31c90040d5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.592212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 705/1405] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:00 2022] POST /identity/v3/users => generated 309 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:00.592212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4ac3-ed25-44c5-82c0-a4e80f93d533 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.598437 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4ac3-ed25-44c5-82c0-a4e80f93d533 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.598999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4ac3-ed25-44c5-82c0-a4e80f93d533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.619747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672b172-1a3b-4597-8c83-31c90040d5b0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.637694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a41c4ac3-ed25-44c5-82c0-a4e80f93d533 None None] MFA Rules not processed for user `a7917d6400b44916b0e0d9cc21d39d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:00.638804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.639466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.639738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.640008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] PATH_INFO: `/v3/projects/0f51f3eb09864c5cb045f7c198e2f2d2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.640721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a41c4ac3-ed25-44c5-82c0-a4e80f93d533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.644108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f51f3eb09864c5cb045f7c198e2f2d2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.646206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.721114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f51f3eb09864c5cb045f7c198e2f2d2', 'request_id': 'req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:00.744998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f51f3eb09864c5cb045f7c198e2f2d2', 'request_id': 'req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:00.772420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f51f3eb09864c5cb045f7c198e2f2d2', 'request_id': 'req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:00.782976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 706/1406] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:00.785450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c4ac3-ed25-44c5-82c0-a4e80f93d533 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.795862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] The token cache is being invalidate because project 0f51f3eb09864c5cb045f7c198e2f2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:00.796183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:00.796919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:00.803115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 701/1407] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:00 2022] DELETE /identity/v3/projects/0f51f3eb09864c5cb045f7c198e2f2d2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:00.804081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.804275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c4ac3-ed25-44c5-82c0-a4e80f93d533 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.805670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.822748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:00.946219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b6c4b-6160-46a8-a233-9cd404844068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.946853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b6c4b-6160-46a8-a233-9cd404844068 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.947483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b6c4b-6160-46a8-a233-9cd404844068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.947761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b6c4b-6160-46a8-a233-9cd404844068 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.950160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b6c4b-6160-46a8-a233-9cd404844068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:00.950402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b6c4b-6160-46a8-a233-9cd404844068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.958165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b6c4b-6160-46a8-a233-9cd404844068 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.976961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a066c30-eccf-4da1-9edc-35af7ac1f9d7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.002386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 707/1408] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:01.003637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12568ed5-db23-490d-8e19-34231c120b6e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.003916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12568ed5-db23-490d-8e19-34231c120b6e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.004195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12568ed5-db23-490d-8e19-34231c120b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.005201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.006399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.006687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.006955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.007753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.009868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.033862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-12568ed5-db23-490d-8e19-34231c120b6e None None] MFA Rules not processed for user `8f151dc283e8498f961b49320130f770`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.034869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 702/1409] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:01.035828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.036840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-12568ed5-db23-490d-8e19-34231c120b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.039809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.069969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353bd41-2d47-4a86-a16b-bd8df061c6ca admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.075133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8059435-eb2d-4f89-b469-d3c7a20a95f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.076138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8059435-eb2d-4f89-b469-d3c7a20a95f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.076567 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8059435-eb2d-4f89-b469-d3c7a20a95f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.076975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8059435-eb2d-4f89-b469-d3c7a20a95f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.077772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8059435-eb2d-4f89-b469-d3c7a20a95f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.081975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8059435-eb2d-4f89-b469-d3c7a20a95f5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.099986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:01.123986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 703/1410] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/projects => generated 363 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:01.127315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e4baf-c76d-4661-898a-a04e5a6a98aa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.127593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e4baf-c76d-4661-898a-a04e5a6a98aa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.127872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e4baf-c76d-4661-898a-a04e5a6a98aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.177147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-441e4baf-c76d-4661-898a-a04e5a6a98aa None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.178909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-441e4baf-c76d-4661-898a-a04e5a6a98aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.184158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 708/1411] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:01.189517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a938db-01f0-492c-890e-fa23576bc9f6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.190303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a938db-01f0-492c-890e-fa23576bc9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.190743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a938db-01f0-492c-890e-fa23576bc9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.260005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d9a938db-01f0-492c-890e-fa23576bc9f6 None None] MFA Rules not processed for user `5318b31b1138460a8ce0a705b0b720a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.261356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d9a938db-01f0-492c-890e-fa23576bc9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.361762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 704/1412] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:01.361762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e4baf-c76d-4661-898a-a04e5a6a98aa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.364024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-441e4baf-c76d-4661-898a-a04e5a6a98aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.389345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e4baf-c76d-4661-898a-a04e5a6a98aa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.400498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d5a8c-7afd-4f6e-a830-13e803f5d55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.400807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d5a8c-7afd-4f6e-a830-13e803f5d55f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.401056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d5a8c-7afd-4f6e-a830-13e803f5d55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.401333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d5a8c-7afd-4f6e-a830-13e803f5d55f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.402511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d5a8c-7afd-4f6e-a830-13e803f5d55f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.404064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d5a8c-7afd-4f6e-a830-13e803f5d55f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.415857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 709/1413] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:01.416824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a938db-01f0-492c-890e-fa23576bc9f6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.441812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a938db-01f0-492c-890e-fa23576bc9f6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.451953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f538d40-b5c9-4c56-a180-ec7f6cee34e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.451953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538d40-b5c9-4c56-a180-ec7f6cee34e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.452240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538d40-b5c9-4c56-a180-ec7f6cee34e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.452537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538d40-b5c9-4c56-a180-ec7f6cee34e5 admin admin] PATH_INFO: `/v3/projects/89ed7902bd004fba996b62fe76a0b248/users/034da4376ee94f95a3329767bb2f2982/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.473237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 705/1414] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:01.480899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d5a8c-7afd-4f6e-a830-13e803f5d55f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.485072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f538d40-b5c9-4c56-a180-ec7f6cee34e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ed7902bd004fba996b62fe76a0b248, user_id=034da4376ee94f95a3329767bb2f2982, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.488330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f538d40-b5c9-4c56-a180-ec7f6cee34e5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.525195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 710/1415] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:01 2022] PUT /identity/v3/projects/89ed7902bd004fba996b62fe76a0b248/users/034da4376ee94f95a3329767bb2f2982/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:01.528109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe34604-f325-4d40-bd73-e858c9b44a15 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.528109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe34604-f325-4d40-bd73-e858c9b44a15 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.528109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe34604-f325-4d40-bd73-e858c9b44a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.591207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4fe34604-f325-4d40-bd73-e858c9b44a15 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.592456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4fe34604-f325-4d40-bd73-e858c9b44a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.651875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d5a8c-7afd-4f6e-a830-13e803f5d55f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.661414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:01.753955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 711/1416] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:01.755442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559e08-692f-4efe-a959-c5506d1eca34 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.755726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559e08-692f-4efe-a959-c5506d1eca34 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.755992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559e08-692f-4efe-a959-c5506d1eca34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.793349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9619f-a935-4e10-a3aa-b7558fe4a674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.793991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9619f-a935-4e10-a3aa-b7558fe4a674 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.794300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9619f-a935-4e10-a3aa-b7558fe4a674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.794581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9619f-a935-4e10-a3aa-b7558fe4a674 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.799299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d7559e08-692f-4efe-a959-c5506d1eca34 None None] MFA Rules not processed for user `5318b31b1138460a8ce0a705b0b720a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.800028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9619f-a935-4e10-a3aa-b7558fe4a674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.800538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d7559e08-692f-4efe-a959-c5506d1eca34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.802104 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9619f-a935-4e10-a3aa-b7558fe4a674 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.852831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 706/1417] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/users => generated 288 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:01.853770 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9619f-a935-4e10-a3aa-b7558fe4a674 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.876907 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9619f-a935-4e10-a3aa-b7558fe4a674 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.888382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:01.889819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4809a-8b05-4a8f-bf3b-21111cb664b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.891647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4809a-8b05-4a8f-bf3b-21111cb664b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.891926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4809a-8b05-4a8f-bf3b-21111cb664b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.893009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4809a-8b05-4a8f-bf3b-21111cb664b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.895424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4809a-8b05-4a8f-bf3b-21111cb664b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.899641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4809a-8b05-4a8f-bf3b-21111cb664b1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.941018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 712/1418] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:01.941880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7559e08-692f-4efe-a959-c5506d1eca34 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.966463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 707/1419] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:01.966463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4809a-8b05-4a8f-bf3b-21111cb664b1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.975881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7559e08-692f-4efe-a959-c5506d1eca34 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.995348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a1e26-8ee9-4a62-b298-694d31687a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.995980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a1e26-8ee9-4a62-b298-694d31687a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.996253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a1e26-8ee9-4a62-b298-694d31687a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.996520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a1e26-8ee9-4a62-b298-694d31687a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.997172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a1e26-8ee9-4a62-b298-694d31687a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.998822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a1e26-8ee9-4a62-b298-694d31687a43 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.012817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4809a-8b05-4a8f-bf3b-21111cb664b1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.026735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 713/1420] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:02.027454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a1e26-8ee9-4a62-b298-694d31687a43 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.028868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.028868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.029960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.029960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.030244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:02.030650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.032609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.089972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a1e26-8ee9-4a62-b298-694d31687a43 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.099648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 708/1421] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:02.100638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.107786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b049-27b5-4de5-abfb-c353e9da9212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.108419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b049-27b5-4de5-abfb-c353e9da9212 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.108694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b049-27b5-4de5-abfb-c353e9da9212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.108979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b049-27b5-4de5-abfb-c353e9da9212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.109610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b049-27b5-4de5-abfb-c353e9da9212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.111439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b049-27b5-4de5-abfb-c353e9da9212 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.113644 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:02.135875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 714/1422] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:02.136454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b049-27b5-4de5-abfb-c353e9da9212 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.164809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b049-27b5-4de5-abfb-c353e9da9212 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.190002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6aa6-a470-453f-985f-4194ea8e0f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.190656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6aa6-a470-453f-985f-4194ea8e0f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.190931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6aa6-a470-453f-985f-4194ea8e0f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.191200 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6aa6-a470-453f-985f-4194ea8e0f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.191847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6aa6-a470-453f-985f-4194ea8e0f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.193444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6aa6-a470-453f-985f-4194ea8e0f79 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.230438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 715/1423] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:02.230438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6aa6-a470-453f-985f-4194ea8e0f79 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.250690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6aa6-a470-453f-985f-4194ea8e0f79 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.262192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648f2ef-ee7d-4a29-a8c4-17bf9c76d92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.262849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648f2ef-ee7d-4a29-a8c4-17bf9c76d92f admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.263396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648f2ef-ee7d-4a29-a8c4-17bf9c76d92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.263669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648f2ef-ee7d-4a29-a8c4-17bf9c76d92f admin admin] PATH_INFO: `/v3/projects/89ed7902bd004fba996b62fe76a0b248/users/034da4376ee94f95a3329767bb2f2982/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.266837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648f2ef-ee7d-4a29-a8c4-17bf9c76d92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ed7902bd004fba996b62fe76a0b248, user_id=034da4376ee94f95a3329767bb2f2982, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.275783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648f2ef-ee7d-4a29-a8c4-17bf9c76d92f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.299369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c62fb-8da2-418b-b87b-38bbfea3d059 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.310738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 716/1424] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:02 2022] PUT /identity/v3/projects/89ed7902bd004fba996b62fe76a0b248/users/034da4376ee94f95a3329767bb2f2982/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:02.311809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648f2ef-ee7d-4a29-a8c4-17bf9c76d92f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.463986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648f2ef-ee7d-4a29-a8c4-17bf9c76d92f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.477496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef1fb-345f-48ac-b38a-e3e7ce3bace6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.478695 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef1fb-345f-48ac-b38a-e3e7ce3bace6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.478887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef1fb-345f-48ac-b38a-e3e7ce3bace6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.480541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef1fb-345f-48ac-b38a-e3e7ce3bace6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.481045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cef1fb-345f-48ac-b38a-e3e7ce3bace6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.485497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cef1fb-345f-48ac-b38a-e3e7ce3bace6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.544440 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:02.552247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 717/1425] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:02 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:02.556023 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef1fb-345f-48ac-b38a-e3e7ce3bace6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.570358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51d172-e6c3-4a05-8dc0-bf8c65178aaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.571545 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51d172-e6c3-4a05-8dc0-bf8c65178aaf service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.572185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51d172-e6c3-4a05-8dc0-bf8c65178aaf service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.572833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51d172-e6c3-4a05-8dc0-bf8c65178aaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.723034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51d172-e6c3-4a05-8dc0-bf8c65178aaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.723693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef1fb-345f-48ac-b38a-e3e7ce3bace6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.724432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51d172-e6c3-4a05-8dc0-bf8c65178aaf service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.734896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068767c-4754-44ea-8167-0f43c4e15f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.741129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068767c-4754-44ea-8167-0f43c4e15f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.741447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068767c-4754-44ea-8167-0f43c4e15f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.741704 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068767c-4754-44ea-8167-0f43c4e15f5b admin admin] PATH_INFO: `/v3/projects/62746cf6231441e984b9adf600852bc9/users/b0354e47e8b542f7b6a7885224c3508f/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.752700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 709/1426] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:02.772980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068767c-4754-44ea-8167-0f43c4e15f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62746cf6231441e984b9adf600852bc9, user_id=b0354e47e8b542f7b6a7885224c3508f, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.775293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51d172-e6c3-4a05-8dc0-bf8c65178aaf service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.775719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068767c-4754-44ea-8167-0f43c4e15f5b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.812970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51d172-e6c3-4a05-8dc0-bf8c65178aaf service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.816542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 718/1427] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:02 2022] PUT /identity/v3/projects/62746cf6231441e984b9adf600852bc9/users/b0354e47e8b542f7b6a7885224c3508f/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:02.829456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68493-6ec5-498f-811d-d1022801cab7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.830874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68493-6ec5-498f-811d-d1022801cab7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.832793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68493-6ec5-498f-811d-d1022801cab7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.841350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 719/1428] 10.222.0.25 () {54 vars in 912 bytes} [Wed Nov 2 07:28:02 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:02.842807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f68493-6ec5-498f-811d-d1022801cab7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.891959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:03.054917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4985adc-cc6a-458a-8747-19745f16b7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.055862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4985adc-cc6a-458a-8747-19745f16b7ee admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.056255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4985adc-cc6a-458a-8747-19745f16b7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.056673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4985adc-cc6a-458a-8747-19745f16b7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.057429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4985adc-cc6a-458a-8747-19745f16b7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.064791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4985adc-cc6a-458a-8747-19745f16b7ee admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.075487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f68493-6ec5-498f-811d-d1022801cab7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.087321 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 710/1429] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:03.089010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4985adc-cc6a-458a-8747-19745f16b7ee admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.093861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cb777-d984-40b1-ac0b-97220f366dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.095168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cb777-d984-40b1-ac0b-97220f366dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.098515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cb777-d984-40b1-ac0b-97220f366dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.098872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cb777-d984-40b1-ac0b-97220f366dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.099549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7cb777-d984-40b1-ac0b-97220f366dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.106162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7cb777-d984-40b1-ac0b-97220f366dc6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.123475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4985adc-cc6a-458a-8747-19745f16b7ee admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.136533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b32b91-16ec-42bc-bdb5-6207852991d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.137286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b32b91-16ec-42bc-bdb5-6207852991d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.137941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b32b91-16ec-42bc-bdb5-6207852991d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.138372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b32b91-16ec-42bc-bdb5-6207852991d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.139196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b32b91-16ec-42bc-bdb5-6207852991d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.141212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b32b91-16ec-42bc-bdb5-6207852991d5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.158811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 720/1430] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:02 2022] POST /identity/v3/users => generated 292 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:03.160171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630a99a-afbb-47c0-bfef-80af6ba850cf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.160870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630a99a-afbb-47c0-bfef-80af6ba850cf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.161153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630a99a-afbb-47c0-bfef-80af6ba850cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.169148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 711/1431] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:03.170222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b32b91-16ec-42bc-bdb5-6207852991d5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.192133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b32b91-16ec-42bc-bdb5-6207852991d5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.202691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0a636-3cf9-4ef4-a765-935f23215f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.203451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a636-3cf9-4ef4-a765-935f23215f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.206368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a636-3cf9-4ef4-a765-935f23215f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.206860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a636-3cf9-4ef4-a765-935f23215f22 admin admin] PATH_INFO: `/v3/projects/d04ac24612bd4188ada9425297e87193/users/edb280de960b4682a1ff2b36956c8cc0/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.241500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2630a99a-afbb-47c0-bfef-80af6ba850cf None None] MFA Rules not processed for user `c4ad0e5567bd452eadd14ed6ab1edc21`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:03.243722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2630a99a-afbb-47c0-bfef-80af6ba850cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.246828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0a636-3cf9-4ef4-a765-935f23215f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04ac24612bd4188ada9425297e87193, user_id=edb280de960b4682a1ff2b36956c8cc0, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.249926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0a636-3cf9-4ef4-a765-935f23215f22 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.290944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 712/1432] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:03 2022] PUT /identity/v3/projects/d04ac24612bd4188ada9425297e87193/users/edb280de960b4682a1ff2b36956c8cc0/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:03.290944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40471d7d-b78a-447b-a64d-024a23339402 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.290944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40471d7d-b78a-447b-a64d-024a23339402 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.290944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40471d7d-b78a-447b-a64d-024a23339402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.321741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-40471d7d-b78a-447b-a64d-024a23339402 None None] MFA Rules not processed for user `0e8717d368d44102bea0eb82fbee8a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:03.324257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-40471d7d-b78a-447b-a64d-024a23339402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.365083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 721/1433] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:28:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:03.366138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630a99a-afbb-47c0-bfef-80af6ba850cf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.391387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630a99a-afbb-47c0-bfef-80af6ba850cf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.458340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 713/1434] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:03.462325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40471d7d-b78a-447b-a64d-024a23339402 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.489981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:03.587013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40471d7d-b78a-447b-a64d-024a23339402 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.592543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406b104-ae7e-4b99-a5ab-a75b8a452eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.593378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b104-ae7e-4b99-a5ab-a75b8a452eba admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.594176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b104-ae7e-4b99-a5ab-a75b8a452eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.594451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b104-ae7e-4b99-a5ab-a75b8a452eba admin admin] PATH_INFO: `/v3/projects/62746cf6231441e984b9adf600852bc9/users/b0354e47e8b542f7b6a7885224c3508f/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.596892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406b104-ae7e-4b99-a5ab-a75b8a452eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=62746cf6231441e984b9adf600852bc9, user_id=b0354e47e8b542f7b6a7885224c3508f, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.599633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406b104-ae7e-4b99-a5ab-a75b8a452eba admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.621762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 714/1435] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:03 2022] PUT /identity/v3/projects/62746cf6231441e984b9adf600852bc9/users/b0354e47e8b542f7b6a7885224c3508f/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:03.623624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9345-9764-4b9a-84ef-1719735d199a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.625402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9345-9764-4b9a-84ef-1719735d199a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.625402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9345-9764-4b9a-84ef-1719735d199a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.690846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d4bc9345-9764-4b9a-84ef-1719735d199a None None] MFA Rules not processed for user `edb280de960b4682a1ff2b36956c8cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:03.692837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d4bc9345-9764-4b9a-84ef-1719735d199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.705842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.706608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.706976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.707314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.708134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.709823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.722064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 722/1436] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:03.725843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.727771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.803071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:03.841111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 715/1437] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:03.845653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679f10-3803-4749-966b-546c731baaea None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.845943 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679f10-3803-4749-966b-546c731baaea None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.846298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679f10-3803-4749-966b-546c731baaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.875393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a09295-15c3-4a01-a02f-4aebf5f4e73a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.936369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-76679f10-3803-4749-966b-546c731baaea None None] MFA Rules not processed for user `b0354e47e8b542f7b6a7885224c3508f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:03.937318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-76679f10-3803-4749-966b-546c731baaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.994587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79496c9-4bfa-4f47-9810-a835aab34f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.995450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79496c9-4bfa-4f47-9810-a835aab34f1d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.995732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79496c9-4bfa-4f47-9810-a835aab34f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.995998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79496c9-4bfa-4f47-9810-a835aab34f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.015960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79496c9-4bfa-4f47-9810-a835aab34f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.017394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79496c9-4bfa-4f47-9810-a835aab34f1d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.035613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 716/1438] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:04.036814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76679f10-3803-4749-966b-546c731baaea None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.040724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 723/1439] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:04.043671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79496c9-4bfa-4f47-9810-a835aab34f1d service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.075549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:04.172456 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76679f10-3803-4749-966b-546c731baaea None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.201561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9a274-8730-42d0-a9db-2d4021988c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.203933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9a274-8730-42d0-a9db-2d4021988c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.204310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9a274-8730-42d0-a9db-2d4021988c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.205942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9a274-8730-42d0-a9db-2d4021988c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.206671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9a274-8730-42d0-a9db-2d4021988c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.207518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79496c9-4bfa-4f47-9810-a835aab34f1d service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.209320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9a274-8730-42d0-a9db-2d4021988c39 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.227352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710374e5-c686-42a1-b501-dbf99c4f44ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.229674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710374e5-c686-42a1-b501-dbf99c4f44ea admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.230057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710374e5-c686-42a1-b501-dbf99c4f44ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.230359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710374e5-c686-42a1-b501-dbf99c4f44ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.232337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710374e5-c686-42a1-b501-dbf99c4f44ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.237222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710374e5-c686-42a1-b501-dbf99c4f44ea admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.246468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 717/1440] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:04.250940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b25cc-a1ac-4704-85f6-cfee9a212a9b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.251494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b25cc-a1ac-4704-85f6-cfee9a212a9b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.251781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b25cc-a1ac-4704-85f6-cfee9a212a9b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.256175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 718/1441] 10.222.0.25 () {54 vars in 912 bytes} [Wed Nov 2 07:28:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:04.258249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b25cc-a1ac-4704-85f6-cfee9a212a9b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.262663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6f5b25cc-a1ac-4704-85f6-cfee9a212a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:04.300118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 724/1442] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:04 2022] POST /identity/v3/projects => generated 357 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:04.301077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710374e5-c686-42a1-b501-dbf99c4f44ea admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.397643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b25cc-a1ac-4704-85f6-cfee9a212a9b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.402701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710374e5-c686-42a1-b501-dbf99c4f44ea admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.422959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7467-a364-4b1d-9c0f-8168469ea720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.423667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7467-a364-4b1d-9c0f-8168469ea720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.423950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7467-a364-4b1d-9c0f-8168469ea720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.424220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7467-a364-4b1d-9c0f-8168469ea720 admin admin] PATH_INFO: `/v3/projects/24f9e146b8a84e679f9aa676584991f4/users/4777c0ca54414b86b4ea09ca4efa018b/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.461631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7467-a364-4b1d-9c0f-8168469ea720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f9e146b8a84e679f9aa676584991f4, user_id=4777c0ca54414b86b4ea09ca4efa018b, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.464714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7467-a364-4b1d-9c0f-8168469ea720 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.503216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 725/1443] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:04 2022] PUT /identity/v3/projects/24f9e146b8a84e679f9aa676584991f4/users/4777c0ca54414b86b4ea09ca4efa018b/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:04.504808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fd1f6-8388-4bd5-8367-6dc104c739b4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.507313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fd1f6-8388-4bd5-8367-6dc104c739b4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.507698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fd1f6-8388-4bd5-8367-6dc104c739b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.524011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:04.531486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba08cdb-cce6-461e-a729-404f9a3ff68c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.535463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba08cdb-cce6-461e-a729-404f9a3ff68c service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.535891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba08cdb-cce6-461e-a729-404f9a3ff68c service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.536272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba08cdb-cce6-461e-a729-404f9a3ff68c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.558982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fc4fd1f6-8388-4bd5-8367-6dc104c739b4 None None] MFA Rules not processed for user `c4ad0e5567bd452eadd14ed6ab1edc21`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:04.562245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fc4fd1f6-8388-4bd5-8367-6dc104c739b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:04.662057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 726/1444] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:28:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:04.663391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fd1f6-8388-4bd5-8367-6dc104c739b4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.668432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba08cdb-cce6-461e-a729-404f9a3ff68c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.670747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba08cdb-cce6-461e-a729-404f9a3ff68c service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.684285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 719/1445] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:04.685950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b648-b7fe-4c85-961d-a42fea13d867 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.686280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b648-b7fe-4c85-961d-a42fea13d867 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.686561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b648-b7fe-4c85-961d-a42fea13d867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.743755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c216b648-b7fe-4c85-961d-a42fea13d867 None None] MFA Rules not processed for user `034da4376ee94f95a3329767bb2f2982`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:04.745377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c216b648-b7fe-4c85-961d-a42fea13d867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:04.753992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:04.873905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fd1f6-8388-4bd5-8367-6dc104c739b4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.876949 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 720/1446] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:04 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:04.878408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d836-804e-46fd-8c7c-66044a34165f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.878797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d836-804e-46fd-8c7c-66044a34165f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.879288 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d836-804e-46fd-8c7c-66044a34165f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.903231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c99799-cee4-485f-8ae2-36c09c319891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.903859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c99799-cee4-485f-8ae2-36c09c319891 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.904131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c99799-cee4-485f-8ae2-36c09c319891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.904393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c99799-cee4-485f-8ae2-36c09c319891 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.905027 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c99799-cee4-485f-8ae2-36c09c319891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.915932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c99799-cee4-485f-8ae2-36c09c319891 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.934879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4779d836-804e-46fd-8c7c-66044a34165f None None] MFA Rules not processed for user `4777c0ca54414b86b4ea09ca4efa018b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:04.937147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4779d836-804e-46fd-8c7c-66044a34165f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:04.956517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 727/1447] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:04 2022] POST /identity/v3/users => generated 285 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:04.956517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c99799-cee4-485f-8ae2-36c09c319891 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.967571 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c99799-cee4-485f-8ae2-36c09c319891 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.972049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087f009-9dfe-4f22-bed1-bb32c0a42b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.972650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087f009-9dfe-4f22-bed1-bb32c0a42b0c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.972923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087f009-9dfe-4f22-bed1-bb32c0a42b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.973194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087f009-9dfe-4f22-bed1-bb32c0a42b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.984947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087f009-9dfe-4f22-bed1-bb32c0a42b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.987604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087f009-9dfe-4f22-bed1-bb32c0a42b0c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.008651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 728/1448] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:05.009547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087f009-9dfe-4f22-bed1-bb32c0a42b0c service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.135521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 721/1449] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:05.137786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779d836-804e-46fd-8c7c-66044a34165f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.141692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087f009-9dfe-4f22-bed1-bb32c0a42b0c service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.153605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.154219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.154461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.154724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.159573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:05.192219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779d836-804e-46fd-8c7c-66044a34165f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.201546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd32ff-998a-48cc-9a8d-188b76a0196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.203303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd32ff-998a-48cc-9a8d-188b76a0196e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.203973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd32ff-998a-48cc-9a8d-188b76a0196e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.204638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd32ff-998a-48cc-9a8d-188b76a0196e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.205622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd32ff-998a-48cc-9a8d-188b76a0196e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.211190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd32ff-998a-48cc-9a8d-188b76a0196e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.258616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 722/1450] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:05.259598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd32ff-998a-48cc-9a8d-188b76a0196e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.261502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:05.280496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd32ff-998a-48cc-9a8d-188b76a0196e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.299311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.299986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.300268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.300548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.301201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.304755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.306405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.307894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.335506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 729/1451] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:05.336400 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.368889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af712dd7-d954-486e-b4a9-406d870eb49e service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.384644 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 723/1452] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:05 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:05.385531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.391432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585c90c-622e-4a2c-aea5-d984b82d65e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.392199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585c90c-622e-4a2c-aea5-d984b82d65e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.394576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585c90c-622e-4a2c-aea5-d984b82d65e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.394947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585c90c-622e-4a2c-aea5-d984b82d65e6 admin admin] PATH_INFO: `/v3/projects/dea86ab8b12345428cd04828c1ea4be3/users/e7a626311e754eb4a0d7dfe7f93ad629/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.396781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:05.427274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585c90c-622e-4a2c-aea5-d984b82d65e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea86ab8b12345428cd04828c1ea4be3, user_id=e7a626311e754eb4a0d7dfe7f93ad629, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.445287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585c90c-622e-4a2c-aea5-d984b82d65e6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.484048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 730/1453] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:05 2022] PUT /identity/v3/projects/dea86ab8b12345428cd04828c1ea4be3/users/e7a626311e754eb4a0d7dfe7f93ad629/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:05.487357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585c90c-622e-4a2c-aea5-d984b82d65e6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.707297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:05.711898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae7d21-3935-4908-bf7a-ca375c63f034 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.735947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585c90c-622e-4a2c-aea5-d984b82d65e6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.757211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc89558-9fca-44b8-94bd-2b76d87d6dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.759803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89558-9fca-44b8-94bd-2b76d87d6dfd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.760340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89558-9fca-44b8-94bd-2b76d87d6dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.761082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89558-9fca-44b8-94bd-2b76d87d6dfd admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.762115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc89558-9fca-44b8-94bd-2b76d87d6dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.774765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc89558-9fca-44b8-94bd-2b76d87d6dfd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.826321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 731/1454] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:05 2022] POST /identity/v3/users => generated 298 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:05.828580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc89558-9fca-44b8-94bd-2b76d87d6dfd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.871569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc89558-9fca-44b8-94bd-2b76d87d6dfd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.879884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.881579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.884580 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.885023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.887757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaccd00-677a-4388-b7dd-a022a03740c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.888556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaccd00-677a-4388-b7dd-a022a03740c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.889109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaccd00-677a-4388-b7dd-a022a03740c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.889744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaccd00-677a-4388-b7dd-a022a03740c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.890231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:05.890788 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaccd00-677a-4388-b7dd-a022a03740c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.894246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaccd00-677a-4388-b7dd-a022a03740c3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.925645 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 732/1455] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:05.931804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb5688-ef6d-43a6-88d7-85059ca4377b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.938904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb5688-ef6d-43a6-88d7-85059ca4377b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.940510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb5688-ef6d-43a6-88d7-85059ca4377b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.957551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.961824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.998753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 724/1456] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:06.011224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:06.039587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365e039-f0d0-4101-8ae0-3c853e8ffe16 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:06.060562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba86b9-0e6d-4516-9449-3f511402fe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:06.061176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba86b9-0e6d-4516-9449-3f511402fe2f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.061460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba86b9-0e6d-4516-9449-3f511402fe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.061792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba86b9-0e6d-4516-9449-3f511402fe2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.062469 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba86b9-0e6d-4516-9449-3f511402fe2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:06.067229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba86b9-0e6d-4516-9449-3f511402fe2f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:06.080417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5ebb5688-ef6d-43a6-88d7-85059ca4377b None None] MFA Rules not processed for user `006ae080f2884719bd4ddabe815d0d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:06.086512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 725/1457] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:06.088977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ebb5688-ef6d-43a6-88d7-85059ca4377b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:06.093022 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba86b9-0e6d-4516-9449-3f511402fe2f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:06.125771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba86b9-0e6d-4516-9449-3f511402fe2f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:06.147337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd419-c233-4306-b1a8-4dfab8046118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:06.148040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd419-c233-4306-b1a8-4dfab8046118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.148319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd419-c233-4306-b1a8-4dfab8046118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.148595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfd419-c233-4306-b1a8-4dfab8046118 admin admin] PATH_INFO: `/v3/projects/dea86ab8b12345428cd04828c1ea4be3/users/e7a626311e754eb4a0d7dfe7f93ad629/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.153585 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd419-c233-4306-b1a8-4dfab8046118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea86ab8b12345428cd04828c1ea4be3, user_id=e7a626311e754eb4a0d7dfe7f93ad629, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:06.156412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfd419-c233-4306-b1a8-4dfab8046118 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:06.204076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 726/1458] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:06 2022] PUT /identity/v3/projects/dea86ab8b12345428cd04828c1ea4be3/users/e7a626311e754eb4a0d7dfe7f93ad629/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:06.205209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd419-c233-4306-b1a8-4dfab8046118 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:06.259588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 733/1459] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:06.261155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc2ec9-6588-4872-881d-b68f13b254e0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.261555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc2ec9-6588-4872-881d-b68f13b254e0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.261953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc2ec9-6588-4872-881d-b68f13b254e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.324357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a3cc2ec9-6588-4872-881d-b68f13b254e0 None None] MFA Rules not processed for user `e7a626311e754eb4a0d7dfe7f93ad629`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:06.328057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3cc2ec9-6588-4872-881d-b68f13b254e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:06.396343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfd419-c233-4306-b1a8-4dfab8046118 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:06.407332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff2875-b236-4b86-885a-072bb237ba32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:06.408060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff2875-b236-4b86-885a-072bb237ba32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.408341 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff2875-b236-4b86-885a-072bb237ba32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.408618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff2875-b236-4b86-885a-072bb237ba32 admin admin] PATH_INFO: `/v3/projects/7ebaac6510dd45698698a4ac28a36f97/users/a9aaac532e324e5ca5bb856c8b3444d7/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.412845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:06.451541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff2875-b236-4b86-885a-072bb237ba32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ebaac6510dd45698698a4ac28a36f97, user_id=a9aaac532e324e5ca5bb856c8b3444d7, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:06.463392 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff2875-b236-4b86-885a-072bb237ba32 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:06.495429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 727/1460] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:06 2022] PUT /identity/v3/projects/7ebaac6510dd45698698a4ac28a36f97/users/a9aaac532e324e5ca5bb856c8b3444d7/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:06.497042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff2875-b236-4b86-885a-072bb237ba32 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:06.533723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 734/1461] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:06 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:06.543412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc2ec9-6588-4872-881d-b68f13b254e0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:06.651168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:06.683534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff2875-b236-4b86-885a-072bb237ba32 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:06.693962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc2ec9-6588-4872-881d-b68f13b254e0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:06.696274 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a55c-6938-4f38-85cb-dce9fa1cfbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:06.697423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a55c-6938-4f38-85cb-dce9fa1cfbf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.697705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a55c-6938-4f38-85cb-dce9fa1cfbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.697979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a55c-6938-4f38-85cb-dce9fa1cfbf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.701457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8a55c-6938-4f38-85cb-dce9fa1cfbf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:06.707155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8a55c-6938-4f38-85cb-dce9fa1cfbf5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:06.743886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 728/1462] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:06.751999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a55c-6938-4f38-85cb-dce9fa1cfbf5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:06.791035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a55c-6938-4f38-85cb-dce9fa1cfbf5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:06.805687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5f212-3228-4e26-80db-1df559ff7bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:06.806576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f212-3228-4e26-80db-1df559ff7bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.811569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f212-3228-4e26-80db-1df559ff7bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.812043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f212-3228-4e26-80db-1df559ff7bdf admin admin] PATH_INFO: `/v3/projects/7ebaac6510dd45698698a4ac28a36f97/users/a9aaac532e324e5ca5bb856c8b3444d7/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.821105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5f212-3228-4e26-80db-1df559ff7bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ebaac6510dd45698698a4ac28a36f97, user_id=a9aaac532e324e5ca5bb856c8b3444d7, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:06.825513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5f212-3228-4e26-80db-1df559ff7bdf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:06.863062 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8a1ab-f8fc-4367-bbf6-4365c6d58587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:06.863701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8a1ab-f8fc-4367-bbf6-4365c6d58587 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.863979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8a1ab-f8fc-4367-bbf6-4365c6d58587 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.864245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8a1ab-f8fc-4367-bbf6-4365c6d58587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.881430 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 729/1463] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:06 2022] PUT /identity/v3/projects/7ebaac6510dd45698698a4ac28a36f97/users/a9aaac532e324e5ca5bb856c8b3444d7/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:06.886354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e4f1e-8a73-4ef9-a1e1-d21f8eb25a64 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.886646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e4f1e-8a73-4ef9-a1e1-d21f8eb25a64 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.886938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e4f1e-8a73-4ef9-a1e1-d21f8eb25a64 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.903107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 730/1464] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:28:06 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:06.913742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c63168-5059-43f4-a95c-68a5bd2bf33f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.914050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c63168-5059-43f4-a95c-68a5bd2bf33f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.915804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c63168-5059-43f4-a95c-68a5bd2bf33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.041092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-84c63168-5059-43f4-a95c-68a5bd2bf33f None None] MFA Rules not processed for user `a9aaac532e324e5ca5bb856c8b3444d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:07.047430 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-84c63168-5059-43f4-a95c-68a5bd2bf33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.089460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8a1ab-f8fc-4367-bbf6-4365c6d58587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:07.092106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8a1ab-f8fc-4367-bbf6-4365c6d58587 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:07.147512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 735/1465] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:07.151261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e81be-7968-4573-a0aa-edc2a5449729 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.159246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e81be-7968-4573-a0aa-edc2a5449729 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.160425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e81be-7968-4573-a0aa-edc2a5449729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.268694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ad9e81be-7968-4573-a0aa-edc2a5449729 None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:07.279728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ad9e81be-7968-4573-a0aa-edc2a5449729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.313800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 731/1466] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:07.366537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c63168-5059-43f4-a95c-68a5bd2bf33f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:07.385843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:07.476015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 736/1467] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:07.477347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e81be-7968-4573-a0aa-edc2a5449729 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:07.480105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ad9e81be-7968-4573-a0aa-edc2a5449729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.518510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c63168-5059-43f4-a95c-68a5bd2bf33f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:07.528094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0097473-666d-4543-a83c-3dfe30fd70a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:07.528570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097473-666d-4543-a83c-3dfe30fd70a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.529316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097473-666d-4543-a83c-3dfe30fd70a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.529316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097473-666d-4543-a83c-3dfe30fd70a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.565040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e81be-7968-4573-a0aa-edc2a5449729 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:07.592664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1263d-67b7-4ec6-923e-c10bf6f936d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:07.593481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1263d-67b7-4ec6-923e-c10bf6f936d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.594219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1263d-67b7-4ec6-923e-c10bf6f936d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.594702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1263d-67b7-4ec6-923e-c10bf6f936d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.716360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0097473-666d-4543-a83c-3dfe30fd70a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:07.718962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0097473-666d-4543-a83c-3dfe30fd70a0 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:07.749881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 732/1468] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:07.751952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f10d-e1f0-4347-94d4-1bbdd8f66801 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.755187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f10d-e1f0-4347-94d4-1bbdd8f66801 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.755668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f10d-e1f0-4347-94d4-1bbdd8f66801 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.760399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 733/1469] 10.222.0.25 () {54 vars in 961 bytes} [Wed Nov 2 07:28:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:07.766673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d899d9b-354d-4f52-8a3e-3bd3239e12da None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.766895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d899d9b-354d-4f52-8a3e-3bd3239e12da None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.767108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d899d9b-354d-4f52-8a3e-3bd3239e12da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.772095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1263d-67b7-4ec6-923e-c10bf6f936d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:07.774142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1263d-67b7-4ec6-923e-c10bf6f936d4 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:07.807884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 737/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:07.826999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9d899d9b-354d-4f52-8a3e-3bd3239e12da None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:07.828285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9d899d9b-354d-4f52-8a3e-3bd3239e12da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.849243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 734/1471] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Nov 2 07:28:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:07.859642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1263d-67b7-4ec6-923e-c10bf6f936d4 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:07.895307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1263d-67b7-4ec6-923e-c10bf6f936d4 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:07.914376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5aa6d-099a-48d8-899f-ce7a2692596d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:07.915465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5aa6d-099a-48d8-899f-ce7a2692596d service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.917680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5aa6d-099a-48d8-899f-ce7a2692596d service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.926747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5aa6d-099a-48d8-899f-ce7a2692596d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.961755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5aa6d-099a-48d8-899f-ce7a2692596d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:07.968864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5aa6d-099a-48d8-899f-ce7a2692596d service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:08.015403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 738/1472] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:09.131564 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec08dd-f49f-4cf9-ac16-305c6a5ae1e9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.132198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec08dd-f49f-4cf9-ac16-305c6a5ae1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.132198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec08dd-f49f-4cf9-ac16-305c6a5ae1e9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.133720 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 735/1473] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:09.156678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91773393-1a4c-454c-97ac-f8e510ff7460 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.157299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91773393-1a4c-454c-97ac-f8e510ff7460 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.157299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91773393-1a4c-454c-97ac-f8e510ff7460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.178875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-91773393-1a4c-454c-97ac-f8e510ff7460 None None] MFA Rules not processed for user `db7ab631609f49249a010b1a8b03e002`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:09.180199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-91773393-1a4c-454c-97ac-f8e510ff7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.191767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 739/1474] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:09.235535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec08dd-f49f-4cf9-ac16-305c6a5ae1e9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.258384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec08dd-f49f-4cf9-ac16-305c6a5ae1e9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.275195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38d1ca-4b4f-4735-ba93-bb70fe691c2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.276166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d1ca-4b4f-4735-ba93-bb70fe691c2d service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.276602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d1ca-4b4f-4735-ba93-bb70fe691c2d service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.277102 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d1ca-4b4f-4735-ba93-bb70fe691c2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.284204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:09.294419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38d1ca-4b4f-4735-ba93-bb70fe691c2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.296527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38d1ca-4b4f-4735-ba93-bb70fe691c2d service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.311603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 736/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:09.485843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91773393-1a4c-454c-97ac-f8e510ff7460 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.583837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91773393-1a4c-454c-97ac-f8e510ff7460 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.589401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.590605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.591096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.602890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] PATH_INFO: `/v3/users/8db17a890cce4b64914e5c883b3d022d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.605376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db17a890cce4b64914e5c883b3d022d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.608801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.703395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db17a890cce4b64914e5c883b3d022d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.707148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db17a890cce4b64914e5c883b3d022d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.714412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:09.736793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db17a890cce4b64914e5c883b3d022d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.748211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db17a890cce4b64914e5c883b3d022d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.758536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-feacfe95-9c6d-45f7-927b-57a47e187bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db17a890cce4b64914e5c883b3d022d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.779824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 740/1476] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:09 2022] DELETE /identity/v3/users/8db17a890cce4b64914e5c883b3d022d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:10.229888 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38d1ca-4b4f-4735-ba93-bb70fe691c2d service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:10.253499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38d1ca-4b4f-4735-ba93-bb70fe691c2d service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:10.334876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47bad-97e8-4015-8f6c-55f0e29bf796 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.335713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47bad-97e8-4015-8f6c-55f0e29bf796 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.336121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47bad-97e8-4015-8f6c-55f0e29bf796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.574977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-94a47bad-97e8-4015-8f6c-55f0e29bf796 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:10.576214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-94a47bad-97e8-4015-8f6c-55f0e29bf796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:10.591964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:10.594236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.594787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.595800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] PATH_INFO: `/v3/users/a7917d6400b44916b0e0d9cc21d39d72` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.599036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7917d6400b44916b0e0d9cc21d39d72)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:10.608221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:10.777508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7917d6400b44916b0e0d9cc21d39d72'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.778863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7917d6400b44916b0e0d9cc21d39d72'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.811948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7917d6400b44916b0e0d9cc21d39d72'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.848449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7917d6400b44916b0e0d9cc21d39d72'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.850259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-053f91c7-c00c-408c-8057-2636d2dda02d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7917d6400b44916b0e0d9cc21d39d72'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.880333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 737/1477] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:10 2022] DELETE /identity/v3/users/a7917d6400b44916b0e0d9cc21d39d72 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:10.885299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c90c53-73ad-481b-aa53-9b7a641b425e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.888168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c90c53-73ad-481b-aa53-9b7a641b425e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.888527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c90c53-73ad-481b-aa53-9b7a641b425e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.931047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 741/1478] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:28:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 597 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:11.004802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a47bad-97e8-4015-8f6c-55f0e29bf796 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:11.006695 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-94a47bad-97e8-4015-8f6c-55f0e29bf796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.037598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-50c90c53-73ad-481b-aa53-9b7a641b425e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:11.041309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-50c90c53-73ad-481b-aa53-9b7a641b425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.078376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a47bad-97e8-4015-8f6c-55f0e29bf796 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:11.347957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 738/1479] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:11.355179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c90c53-73ad-481b-aa53-9b7a641b425e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:11.365591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:11.367217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-50c90c53-73ad-481b-aa53-9b7a641b425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.368349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.368811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.369705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] PATH_INFO: `/v3/projects/53ab18099672426e826cab1501379935` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.372883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ab18099672426e826cab1501379935)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:11.377455 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:11.424202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:11.515049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:11.533537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ab18099672426e826cab1501379935', 'request_id': 'req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.551661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ab18099672426e826cab1501379935', 'request_id': 'req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.580133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ab18099672426e826cab1501379935', 'request_id': 'req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.599077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c90c53-73ad-481b-aa53-9b7a641b425e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:11.619563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] The token cache is being invalidate because project 53ab18099672426e826cab1501379935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:11.621429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.626541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.629570 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 742/1480] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:11 2022] DELETE /identity/v3/projects/53ab18099672426e826cab1501379935 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:11.637461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:11.647120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.796411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:11.797622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.798431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.799525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.804121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.869829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24283a2-50fc-4ab7-b28a-f2e1f54936fb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:11.891066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:11.891835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.892103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.892367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.900231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.975027 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:11.976634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:11.996886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 739/1481] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:28:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:28:11.998834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:12.001881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.045901 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:12.045901 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:12.068352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 743/1482] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:12.095919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:12.148745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a83fd-3570-4302-8d79-c226ef3176cb service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:12.168305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:12.169696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.170753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.171085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] PATH_INFO: `/v3/users/034da4376ee94f95a3329767bb2f2982` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.173814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=034da4376ee94f95a3329767bb2f2982)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:12.179751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:12.300377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034da4376ee94f95a3329767bb2f2982'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.301961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034da4376ee94f95a3329767bb2f2982'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.330615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034da4376ee94f95a3329767bb2f2982'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.350025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034da4376ee94f95a3329767bb2f2982'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.351879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034da4376ee94f95a3329767bb2f2982'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.446698 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 740/1483] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:11 2022] DELETE /identity/v3/users/034da4376ee94f95a3329767bb2f2982 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:13.592840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.595793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:13.647693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77c1b9-4f03-402d-b238-a32adad439f2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.916706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.916947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.917775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.918153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] PATH_INFO: `/v3/projects/74520c9f0a9d48d28ca1b0a35a09bbff` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.921463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74520c9f0a9d48d28ca1b0a35a09bbff)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.926115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.940050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.942118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.016030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:14.080336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74520c9f0a9d48d28ca1b0a35a09bbff', 'request_id': 'req-a3f43891-5deb-4697-b36b-c03fba638fc8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.096256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74520c9f0a9d48d28ca1b0a35a09bbff', 'request_id': 'req-a3f43891-5deb-4697-b36b-c03fba638fc8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.120885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74520c9f0a9d48d28ca1b0a35a09bbff', 'request_id': 'req-a3f43891-5deb-4697-b36b-c03fba638fc8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.148771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] The token cache is being invalidate because project 74520c9f0a9d48d28ca1b0a35a09bbff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:14.149100 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.152004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f43891-5deb-4697-b36b-c03fba638fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.155079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 744/1484] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:13 2022] DELETE /identity/v3/projects/74520c9f0a9d48d28ca1b0a35a09bbff => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:14.157794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124379a2-1dd7-4da2-96bb-416f6ae0c907 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.158468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124379a2-1dd7-4da2-96bb-416f6ae0c907 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.159224 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124379a2-1dd7-4da2-96bb-416f6ae0c907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.175120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88d63-4deb-45e5-97f3-3b9581f63d89 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:14.214249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-124379a2-1dd7-4da2-96bb-416f6ae0c907 None None] MFA Rules not processed for user `c4ad0e5567bd452eadd14ed6ab1edc21`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:14.217824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-124379a2-1dd7-4da2-96bb-416f6ae0c907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.343095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:14.344289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.344609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.347852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.354540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.364367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 745/1485] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Nov 2 07:28:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:14.365956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124379a2-1dd7-4da2-96bb-416f6ae0c907 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:14.373700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-124379a2-1dd7-4da2-96bb-416f6ae0c907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.440550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:14.522772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124379a2-1dd7-4da2-96bb-416f6ae0c907 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:14.538693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:14.540530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:14.541193 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.541322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:14.541472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.541743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] PATH_INFO: `/v3/users/b0354e47e8b542f7b6a7885224c3508f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.543431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0354e47e8b542f7b6a7885224c3508f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:14.545204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:14.553815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 741/1486] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:14.555462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:14.557036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.572616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3eb86-f2e5-40e3-9bc8-54fbccb693de service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:14.577264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:14.578051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.578349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.578839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.587559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:14.589675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:14.605679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 742/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:14.639646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0354e47e8b542f7b6a7885224c3508f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.642135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0354e47e8b542f7b6a7885224c3508f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.671080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:14.676731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0354e47e8b542f7b6a7885224c3508f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.697042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0354e47e8b542f7b6a7885224c3508f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.698948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2df00262-9e41-411c-b7a7-c754f562e569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0354e47e8b542f7b6a7885224c3508f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:14.740198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 746/1488] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:14 2022] DELETE /identity/v3/users/b0354e47e8b542f7b6a7885224c3508f => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:15.478565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.480793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:15.544442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d742fa-b092-444c-bf87-96add81a435b service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.772157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.772935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.773379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.774390 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] PATH_INFO: `/v3/projects/89ed7902bd004fba996b62fe76a0b248` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.779660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ed7902bd004fba996b62fe76a0b248)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.783013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.924800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ed7902bd004fba996b62fe76a0b248', 'request_id': 'req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:15.940977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ed7902bd004fba996b62fe76a0b248', 'request_id': 'req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:15.958934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ed7902bd004fba996b62fe76a0b248', 'request_id': 'req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:15.988725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] The token cache is being invalidate because project 89ed7902bd004fba996b62fe76a0b248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:15.990520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:15.992504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d79be6d-d8d0-419f-a62b-bdafdf4b6b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:15.998169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 743/1489] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:15 2022] DELETE /identity/v3/projects/89ed7902bd004fba996b62fe76a0b248 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:16.004919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71baa1e-cbe5-4965-a405-09fa50eee851 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.006824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71baa1e-cbe5-4965-a405-09fa50eee851 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.007555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71baa1e-cbe5-4965-a405-09fa50eee851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.047428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a71baa1e-cbe5-4965-a405-09fa50eee851 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:16.051650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a71baa1e-cbe5-4965-a405-09fa50eee851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:16.318499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 747/1490] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:16.320121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3111-937f-48f5-9635-c786f193bc10 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.320417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3111-937f-48f5-9635-c786f193bc10 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.320697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3111-937f-48f5-9635-c786f193bc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.422855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-476f3111-937f-48f5-9635-c786f193bc10 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:16.436108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-476f3111-937f-48f5-9635-c786f193bc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:16.478939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 744/1491] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:16.483207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71baa1e-cbe5-4965-a405-09fa50eee851 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.497292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71baa1e-cbe5-4965-a405-09fa50eee851 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.536278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:16.539592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.539882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.540162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.540839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:16.541434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:16.542796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:16.577734 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 748/1492] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:16.579695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f3111-937f-48f5-9635-c786f193bc10 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.598856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f3111-937f-48f5-9635-c786f193bc10 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.614057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:16.619794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.620164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.620486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.621242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:16.623410 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:16.704892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 745/1493] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:16 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:16.706780 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.745986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.749716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:16.757843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859ed76-dc54-42b5-ab41-742312bcdd7b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.805866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:17.011353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f453f53-ec1c-4edd-bfb0-9041173933c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:17.013549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453f53-ec1c-4edd-bfb0-9041173933c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.014696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453f53-ec1c-4edd-bfb0-9041173933c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.015204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f453f53-ec1c-4edd-bfb0-9041173933c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.017779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f453f53-ec1c-4edd-bfb0-9041173933c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:17.024338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f453f53-ec1c-4edd-bfb0-9041173933c2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:17.092450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b4ef3-f5e9-4268-948e-20451614dde5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:17.111099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:17.112061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.112538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 749/1494] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:16 2022] POST /identity/v3/users => generated 291 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:17.122405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.124191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acdf4d-443d-43cc-a0d1-e34d4d151d06 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.125285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] PATH_INFO: `/v3/projects/62746cf6231441e984b9adf600852bc9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.126511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acdf4d-443d-43cc-a0d1-e34d4d151d06 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.127958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acdf4d-443d-43cc-a0d1-e34d4d151d06 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.131739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62746cf6231441e984b9adf600852bc9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:17.136033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 750/1495] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:28:17 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:17.138955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acdf4d-443d-43cc-a0d1-e34d4d151d06 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:17.142370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:17.219093 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acdf4d-443d-43cc-a0d1-e34d4d151d06 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:17.251967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152003ee-d987-4041-9a24-89734cb3265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:17.252941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152003ee-d987-4041-9a24-89734cb3265a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.252941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152003ee-d987-4041-9a24-89734cb3265a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.262074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152003ee-d987-4041-9a24-89734cb3265a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.262074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152003ee-d987-4041-9a24-89734cb3265a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:17.269393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152003ee-d987-4041-9a24-89734cb3265a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:17.319489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 751/1496] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:17.321047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be348ba5-0193-4526-8406-06d20a846d4b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.321324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be348ba5-0193-4526-8406-06d20a846d4b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.325161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be348ba5-0193-4526-8406-06d20a846d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.390891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62746cf6231441e984b9adf600852bc9', 'request_id': 'req-fa67f851-f899-45e4-bcc5-be5952c0ee06'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:17.409205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62746cf6231441e984b9adf600852bc9', 'request_id': 'req-fa67f851-f899-45e4-bcc5-be5952c0ee06'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:17.415537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-be348ba5-0193-4526-8406-06d20a846d4b None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:17.423824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be348ba5-0193-4526-8406-06d20a846d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:17.448404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62746cf6231441e984b9adf600852bc9', 'request_id': 'req-fa67f851-f899-45e4-bcc5-be5952c0ee06'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:17.514189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] The token cache is being invalidate because project 62746cf6231441e984b9adf600852bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:17.514189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:17.516898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:17.525309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 746/1497] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:16 2022] DELETE /identity/v3/projects/62746cf6231441e984b9adf600852bc9 => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:17.526292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:17.550579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:17.655959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:17.768067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 752/1498] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:17.772097 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d0c6-d316-4a19-96c4-76d7bad420be None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.772392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d0c6-d316-4a19-96c4-76d7bad420be None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.772668 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d0c6-d316-4a19-96c4-76d7bad420be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.806245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67f851-f899-45e4-bcc5-be5952c0ee06 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:17.817269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80036724-69df-406c-a15d-2378d921b204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:17.817984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036724-69df-406c-a15d-2378d921b204 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.824211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036724-69df-406c-a15d-2378d921b204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.824211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036724-69df-406c-a15d-2378d921b204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.824449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80036724-69df-406c-a15d-2378d921b204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:17.832733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80036724-69df-406c-a15d-2378d921b204 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:17.860125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 747/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:17.865324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80036724-69df-406c-a15d-2378d921b204 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:17.880821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ff12d0c6-d316-4a19-96c4-76d7bad420be None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:17.882068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ff12d0c6-d316-4a19-96c4-76d7bad420be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:17.900284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80036724-69df-406c-a15d-2378d921b204 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:17.917388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc071c57-70a0-4034-aef1-acccba5e6870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:17.918239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc071c57-70a0-4034-aef1-acccba5e6870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.918730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc071c57-70a0-4034-aef1-acccba5e6870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.919055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc071c57-70a0-4034-aef1-acccba5e6870 admin admin] PATH_INFO: `/v3/projects/8c7df5dfbc2b46c9b7492f9fa153b80e/users/1277eddad21746768f5ed0970ffdb0eb/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.963871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 753/1500] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:17.967026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d8573-6c69-4a95-a2c4-26da1559d897 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.968350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d8573-6c69-4a95-a2c4-26da1559d897 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.969348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d8573-6c69-4a95-a2c4-26da1559d897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.974881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc071c57-70a0-4034-aef1-acccba5e6870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7df5dfbc2b46c9b7492f9fa153b80e, user_id=1277eddad21746768f5ed0970ffdb0eb, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:17.979819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc071c57-70a0-4034-aef1-acccba5e6870 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.037316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 748/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:17 2022] PUT /identity/v3/projects/8c7df5dfbc2b46c9b7492f9fa153b80e/users/1277eddad21746768f5ed0970ffdb0eb/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:18.048928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6db62-d0ad-436b-b8c9-a3713d00aab7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.049621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6db62-d0ad-436b-b8c9-a3713d00aab7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.050053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6db62-d0ad-436b-b8c9-a3713d00aab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.069772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8b1d8573-6c69-4a95-a2c4-26da1559d897 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:18.071079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8b1d8573-6c69-4a95-a2c4-26da1559d897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:18.139150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:18.151296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8bc6db62-d0ad-436b-b8c9-a3713d00aab7 None None] MFA Rules not processed for user `1277eddad21746768f5ed0970ffdb0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:18.153685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8bc6db62-d0ad-436b-b8c9-a3713d00aab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:18.239849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 754/1502] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:18.249867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d8573-6c69-4a95-a2c4-26da1559d897 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.288895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d8573-6c69-4a95-a2c4-26da1559d897 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.312248 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.315078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.315380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.315641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.316407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:18.316750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.325681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.347842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 749/1503] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:18.352550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6db62-d0ad-436b-b8c9-a3713d00aab7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.368184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6db62-d0ad-436b-b8c9-a3713d00aab7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.374246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4891719-8ecd-4bb2-a95f-5607204064e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.374982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891719-8ecd-4bb2-a95f-5607204064e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.375330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891719-8ecd-4bb2-a95f-5607204064e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.375732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891719-8ecd-4bb2-a95f-5607204064e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.376550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4891719-8ecd-4bb2-a95f-5607204064e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.379935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4891719-8ecd-4bb2-a95f-5607204064e4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.385598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 755/1504] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:18.389086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.419729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43a614-4f1e-4c39-9c1d-15852b2e16ac admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.440198 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8916699-f8c4-41af-bc2b-babaec88f3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.441045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916699-f8c4-41af-bc2b-babaec88f3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.441706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916699-f8c4-41af-bc2b-babaec88f3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.442263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8916699-f8c4-41af-bc2b-babaec88f3a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.443097 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8916699-f8c4-41af-bc2b-babaec88f3a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.446482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8916699-f8c4-41af-bc2b-babaec88f3a8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.471442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 750/1505] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:18 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:18.475789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4891719-8ecd-4bb2-a95f-5607204064e4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.543195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 756/1506] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:18 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:18.557143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8916699-f8c4-41af-bc2b-babaec88f3a8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.611447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4891719-8ecd-4bb2-a95f-5607204064e4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.707263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8916699-f8c4-41af-bc2b-babaec88f3a8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.718059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb220d-fffe-4f67-b594-c8683ce6f0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.719725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb220d-fffe-4f67-b594-c8683ce6f0df admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.733943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb220d-fffe-4f67-b594-c8683ce6f0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.737973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb220d-fffe-4f67-b594-c8683ce6f0df admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.738987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb220d-fffe-4f67-b594-c8683ce6f0df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.742457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb220d-fffe-4f67-b594-c8683ce6f0df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.753496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdb1ff-0a11-4868-89e0-44713140644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.754703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdb1ff-0a11-4868-89e0-44713140644c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.755839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdb1ff-0a11-4868-89e0-44713140644c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.756353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdb1ff-0a11-4868-89e0-44713140644c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.758461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdb1ff-0a11-4868-89e0-44713140644c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.768187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdb1ff-0a11-4868-89e0-44713140644c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.785679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 757/1507] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:18 2022] POST /identity/v3/users => generated 283 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:18.790321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb220d-fffe-4f67-b594-c8683ce6f0df admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.812999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb220d-fffe-4f67-b594-c8683ce6f0df admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.819796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3e4d6-d94d-438c-831d-b700032394bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.820501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3e4d6-d94d-438c-831d-b700032394bf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.820836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3e4d6-d94d-438c-831d-b700032394bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.821239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3e4d6-d94d-438c-831d-b700032394bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.822148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3e4d6-d94d-438c-831d-b700032394bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.824075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 751/1508] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:18 2022] POST /identity/v3/users => generated 298 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:18.826798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3e4d6-d94d-438c-831d-b700032394bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.828056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdb1ff-0a11-4868-89e0-44713140644c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.844556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 758/1509] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:18.850338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3e4d6-d94d-438c-831d-b700032394bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.868275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdb1ff-0a11-4868-89e0-44713140644c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.868958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3e4d6-d94d-438c-831d-b700032394bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.875861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.877686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.878117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.878529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.879517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.886419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f192c3-54e4-43a3-ad8d-06af49eb28b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.887785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.888026 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f192c3-54e4-43a3-ad8d-06af49eb28b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.888407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f192c3-54e4-43a3-ad8d-06af49eb28b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.889237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f192c3-54e4-43a3-ad8d-06af49eb28b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.914778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 759/1510] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:18.915463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f192c3-54e4-43a3-ad8d-06af49eb28b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.916303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.917649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f192c3-54e4-43a3-ad8d-06af49eb28b8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.922126 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:18.941953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 752/1511] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:18.944110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f192c3-54e4-43a3-ad8d-06af49eb28b8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.945351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:19.011549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f192c3-54e4-43a3-ad8d-06af49eb28b8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:19.023784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343219da-622b-440d-b328-14e78a223ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:19.036331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343219da-622b-440d-b328-14e78a223ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:19.037268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343219da-622b-440d-b328-14e78a223ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:19.040145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343219da-622b-440d-b328-14e78a223ad1 admin admin] PATH_INFO: `/v3/projects/373652300a964d34a75002c71df7dd93/users/f31d0ae8e82742deb83f850f8b55ae80/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:19.047318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:19.085549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343219da-622b-440d-b328-14e78a223ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=373652300a964d34a75002c71df7dd93, user_id=f31d0ae8e82742deb83f850f8b55ae80, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:19.090479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343219da-622b-440d-b328-14e78a223ad1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:19.130507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 753/1512] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:18 2022] PUT /identity/v3/projects/373652300a964d34a75002c71df7dd93/users/f31d0ae8e82742deb83f850f8b55ae80/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:19.130507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343219da-622b-440d-b328-14e78a223ad1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:19.172640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac67019-6924-4b5d-83fb-d7b43545faa3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:19.257017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343219da-622b-440d-b328-14e78a223ad1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:19.269493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d6daf-a383-4115-8566-4458f5a27c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:19.270480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d6daf-a383-4115-8566-4458f5a27c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:19.270596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d6daf-a383-4115-8566-4458f5a27c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:19.271017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d6daf-a383-4115-8566-4458f5a27c54 admin admin] PATH_INFO: `/v3/projects/5e186d48576e4d429cdd63b1fb07c186/users/ed9f247b208a4ae59e0d9cc843cf4272/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:19.297762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d6daf-a383-4115-8566-4458f5a27c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e186d48576e4d429cdd63b1fb07c186, user_id=ed9f247b208a4ae59e0d9cc843cf4272, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:19.300853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d6daf-a383-4115-8566-4458f5a27c54 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:19.336879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 754/1513] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:19 2022] PUT /identity/v3/projects/5e186d48576e4d429cdd63b1fb07c186/users/ed9f247b208a4ae59e0d9cc843cf4272/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:19.336879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f97c59-3982-476e-ad89-076d989aeb88 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:19.336879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f97c59-3982-476e-ad89-076d989aeb88 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:19.336879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f97c59-3982-476e-ad89-076d989aeb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:19.383313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4dbc4-ed34-4579-b78c-e2f207a5749c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:19.385021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4dbc4-ed34-4579-b78c-e2f207a5749c service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:19.385846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4dbc4-ed34-4579-b78c-e2f207a5749c service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:19.386918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4dbc4-ed34-4579-b78c-e2f207a5749c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:19.407263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d5f97c59-3982-476e-ad89-076d989aeb88 None None] MFA Rules not processed for user `f31d0ae8e82742deb83f850f8b55ae80`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:19.408999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d5f97c59-3982-476e-ad89-076d989aeb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:19.411821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4dbc4-ed34-4579-b78c-e2f207a5749c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:19.415244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4dbc4-ed34-4579-b78c-e2f207a5749c service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:19.471835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 760/1514] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:19.480656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4dbc4-ed34-4579-b78c-e2f207a5749c service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:19.489613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 755/1515] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:19.497825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f97c59-3982-476e-ad89-076d989aeb88 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:19.514531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4dbc4-ed34-4579-b78c-e2f207a5749c service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:19.699732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:19.714733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f97c59-3982-476e-ad89-076d989aeb88 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:19.735792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fb315-c3bc-46b2-aeed-f40f5357cb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:19.736692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fb315-c3bc-46b2-aeed-f40f5357cb81 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:19.737127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fb315-c3bc-46b2-aeed-f40f5357cb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:19.737579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fb315-c3bc-46b2-aeed-f40f5357cb81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:19.740597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fb315-c3bc-46b2-aeed-f40f5357cb81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:19.744420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fb315-c3bc-46b2-aeed-f40f5357cb81 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:19.747015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22e1bb-dca7-47cc-b6ed-16418c09af89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:19.747733 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22e1bb-dca7-47cc-b6ed-16418c09af89 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:19.748884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22e1bb-dca7-47cc-b6ed-16418c09af89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:19.750515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22e1bb-dca7-47cc-b6ed-16418c09af89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:19.751339 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22e1bb-dca7-47cc-b6ed-16418c09af89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:19.759810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22e1bb-dca7-47cc-b6ed-16418c09af89 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:19.823470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 761/1516] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:19.826321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22e1bb-dca7-47cc-b6ed-16418c09af89 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:19.832336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 756/1517] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:19 2022] POST /identity/v3/projects => generated 367 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:19.837457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fb315-c3bc-46b2-aeed-f40f5357cb81 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:19.853179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22e1bb-dca7-47cc-b6ed-16418c09af89 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:19.915003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:20.032979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fb315-c3bc-46b2-aeed-f40f5357cb81 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.058624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575b5c8-5c69-474a-925b-1e3eeff4da02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.059293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b5c8-5c69-474a-925b-1e3eeff4da02 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.059610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b5c8-5c69-474a-925b-1e3eeff4da02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.059863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b5c8-5c69-474a-925b-1e3eeff4da02 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.059967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3b1b4-86a1-4227-bd25-6f7e2d96e5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.060534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575b5c8-5c69-474a-925b-1e3eeff4da02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.061325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3b1b4-86a1-4227-bd25-6f7e2d96e5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.062804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575b5c8-5c69-474a-925b-1e3eeff4da02 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.076524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3b1b4-86a1-4227-bd25-6f7e2d96e5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.077791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3b1b4-86a1-4227-bd25-6f7e2d96e5fb admin admin] PATH_INFO: `/v3/projects/5e186d48576e4d429cdd63b1fb07c186/users/ed9f247b208a4ae59e0d9cc843cf4272/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.086290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3b1b4-86a1-4227-bd25-6f7e2d96e5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e186d48576e4d429cdd63b1fb07c186, user_id=ed9f247b208a4ae59e0d9cc843cf4272, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.101278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3b1b4-86a1-4227-bd25-6f7e2d96e5fb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.113818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 757/1518] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:19 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:20.115242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575b5c8-5c69-474a-925b-1e3eeff4da02 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.138514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575b5c8-5c69-474a-925b-1e3eeff4da02 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.143429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 762/1519] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:19 2022] PUT /identity/v3/projects/5e186d48576e4d429cdd63b1fb07c186/users/ed9f247b208a4ae59e0d9cc843cf4272/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:20.154317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4dcca-7c6b-4676-b537-71e290698445 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.154945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4dcca-7c6b-4676-b537-71e290698445 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.156022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4dcca-7c6b-4676-b537-71e290698445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.238462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d3b4dcca-7c6b-4676-b537-71e290698445 None None] MFA Rules not processed for user `ed9f247b208a4ae59e0d9cc843cf4272`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:20.241889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d3b4dcca-7c6b-4676-b537-71e290698445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:20.271008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9a5d-6123-4792-8877-c8d4688d75ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.271646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9a5d-6123-4792-8877-c8d4688d75ca admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.271976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9a5d-6123-4792-8877-c8d4688d75ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.272294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9a5d-6123-4792-8877-c8d4688d75ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.272953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c9a5d-6123-4792-8877-c8d4688d75ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.274640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c9a5d-6123-4792-8877-c8d4688d75ca admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.304754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 758/1520] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:20.306942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9a5d-6123-4792-8877-c8d4688d75ca admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.329000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9a5d-6123-4792-8877-c8d4688d75ca admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.352147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.359387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.360050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.361052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] PATH_INFO: `/v3/projects/6ee4ff0a7f3844b68f1e946913ae1c1e/users/5917cdfac9b242e2bd6bf3b59fa38cbf/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.426116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee4ff0a7f3844b68f1e946913ae1c1e, user_id=5917cdfac9b242e2bd6bf3b59fa38cbf, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.434431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.460400 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 763/1521] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:20.472709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 759/1522] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:20 2022] PUT /identity/v3/projects/6ee4ff0a7f3844b68f1e946913ae1c1e/users/5917cdfac9b242e2bd6bf3b59fa38cbf/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:20.477336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4dcca-7c6b-4676-b537-71e290698445 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.483384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.485863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:20.517301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4dcca-7c6b-4676-b537-71e290698445 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.620303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:20.668052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf47762-fa48-4480-8a6c-8410d43f5dde admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.673826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.674796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.675802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.676673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.687246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae6949-e21b-4e9d-a317-0993aa52bf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.688003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae6949-e21b-4e9d-a317-0993aa52bf90 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.688654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae6949-e21b-4e9d-a317-0993aa52bf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.688945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae6949-e21b-4e9d-a317-0993aa52bf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.689673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae6949-e21b-4e9d-a317-0993aa52bf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.691418 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae6949-e21b-4e9d-a317-0993aa52bf90 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.716888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 764/1523] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:20.717985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae6949-e21b-4e9d-a317-0993aa52bf90 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.757229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae6949-e21b-4e9d-a317-0993aa52bf90 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.765409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa6448-ad8c-47bc-af68-ced8da24280e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.766115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6448-ad8c-47bc-af68-ced8da24280e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.766506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6448-ad8c-47bc-af68-ced8da24280e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.766687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6448-ad8c-47bc-af68-ced8da24280e admin admin] PATH_INFO: `/v3/projects/6ee4ff0a7f3844b68f1e946913ae1c1e/users/5917cdfac9b242e2bd6bf3b59fa38cbf/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.771533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa6448-ad8c-47bc-af68-ced8da24280e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee4ff0a7f3844b68f1e946913ae1c1e, user_id=5917cdfac9b242e2bd6bf3b59fa38cbf, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.774233 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa6448-ad8c-47bc-af68-ced8da24280e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.800596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 765/1524] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:20 2022] PUT /identity/v3/projects/6ee4ff0a7f3844b68f1e946913ae1c1e/users/5917cdfac9b242e2bd6bf3b59fa38cbf/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:20.812183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cc1da-88b5-4150-99d1-e0b1608d91b7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.812465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cc1da-88b5-4150-99d1-e0b1608d91b7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.812737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cc1da-88b5-4150-99d1-e0b1608d91b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.867403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-039cc1da-88b5-4150-99d1-e0b1608d91b7 None None] MFA Rules not processed for user `5917cdfac9b242e2bd6bf3b59fa38cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:20.868631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-039cc1da-88b5-4150-99d1-e0b1608d91b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:20.920812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:20.944076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.948549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.971991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 760/1525] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:20.995234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.997489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.068349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e913d3a-e8b0-401d-ba55-92565a1d4e5f service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.088953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 766/1526] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:21.127937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cc1da-88b5-4150-99d1-e0b1608d91b7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.339828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.346966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.351092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.351526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] PATH_INFO: `/v3/users/4777c0ca54414b86b4ea09ca4efa018b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.353966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4777c0ca54414b86b4ea09ca4efa018b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.359343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.368039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cc1da-88b5-4150-99d1-e0b1608d91b7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.389909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d425271-6eaf-443a-93fa-e8f537a55ec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.395963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d425271-6eaf-443a-93fa-e8f537a55ec7 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.396778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d425271-6eaf-443a-93fa-e8f537a55ec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.398340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d425271-6eaf-443a-93fa-e8f537a55ec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.427579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d425271-6eaf-443a-93fa-e8f537a55ec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.430635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d425271-6eaf-443a-93fa-e8f537a55ec7 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.472704 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 767/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:21.492377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4777c0ca54414b86b4ea09ca4efa018b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.496587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4777c0ca54414b86b4ea09ca4efa018b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.528488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4777c0ca54414b86b4ea09ca4efa018b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.545740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4777c0ca54414b86b4ea09ca4efa018b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.549620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4777c0ca54414b86b4ea09ca4efa018b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.580444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 761/1528] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:20 2022] DELETE /identity/v3/users/4777c0ca54414b86b4ea09ca4efa018b => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:21.580674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8346ea4-7d11-4daf-a92e-04a9945a0265 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.580674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8346ea4-7d11-4daf-a92e-04a9945a0265 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.580674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8346ea4-7d11-4daf-a92e-04a9945a0265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.599252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.600688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.641982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:21.661110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f8346ea4-7d11-4daf-a92e-04a9945a0265 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:21.666565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f8346ea4-7d11-4daf-a92e-04a9945a0265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.736806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6c37d-c7f0-464f-9e1d-ebbb774bd25b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.750066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.751069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.751330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.751524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.754183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.871665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 768/1529] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:21.874391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.876228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.887402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 762/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:21.909964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8346ea4-7d11-4daf-a92e-04a9945a0265 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.916837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f8346ea4-7d11-4daf-a92e-04a9945a0265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.051077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:22.053215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.066273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8346ea4-7d11-4daf-a92e-04a9945a0265 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.074753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b305ae-6e2e-459d-a4cf-d0927cb410b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.075457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b305ae-6e2e-459d-a4cf-d0927cb410b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.075923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b305ae-6e2e-459d-a4cf-d0927cb410b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.076085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b305ae-6e2e-459d-a4cf-d0927cb410b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.083106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358eb45a-985a-4717-9030-3b411efced44 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.087725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09119a-6980-4f9a-8014-16bb7550e9a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.088344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09119a-6980-4f9a-8014-16bb7550e9a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.088612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09119a-6980-4f9a-8014-16bb7550e9a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.088879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09119a-6980-4f9a-8014-16bb7550e9a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.091132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2e09119a-6980-4f9a-8014-16bb7550e9a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.109644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:22.118277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b305ae-6e2e-459d-a4cf-d0927cb410b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.119861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b305ae-6e2e-459d-a4cf-d0927cb410b4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.166942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 769/1531] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:22.189106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09119a-6980-4f9a-8014-16bb7550e9a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.191725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09119a-6980-4f9a-8014-16bb7550e9a7 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.214920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 763/1532] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:23.090742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d767e0-9e2f-4b8f-aed9-7d03421097b3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.091212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d767e0-9e2f-4b8f-aed9-7d03421097b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.091343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d767e0-9e2f-4b8f-aed9-7d03421097b3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.093183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 770/1533] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:23.124809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2a1a-ced5-4275-ac7a-85ab3ed676b0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.125639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2a1a-ced5-4275-ac7a-85ab3ed676b0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.125639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2a1a-ced5-4275-ac7a-85ab3ed676b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.158864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d767e0-9e2f-4b8f-aed9-7d03421097b3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.161958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-17d767e0-9e2f-4b8f-aed9-7d03421097b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.194319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ea1b2a1a-ced5-4275-ac7a-85ab3ed676b0 None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:23.194319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ea1b2a1a-ced5-4275-ac7a-85ab3ed676b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.225142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d767e0-9e2f-4b8f-aed9-7d03421097b3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.232979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.234162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.234636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.235177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] PATH_INFO: `/v3/projects/24f9e146b8a84e679f9aa676584991f4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.237523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f9e146b8a84e679f9aa676584991f4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.240032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.289544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:23.306358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f9e146b8a84e679f9aa676584991f4', 'request_id': 'req-ea67c891-7fca-4194-bbad-26ce3b0a2155'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.331234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f9e146b8a84e679f9aa676584991f4', 'request_id': 'req-ea67c891-7fca-4194-bbad-26ce3b0a2155'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.333041 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 764/1534] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:23.341773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f9e146b8a84e679f9aa676584991f4', 'request_id': 'req-ea67c891-7fca-4194-bbad-26ce3b0a2155'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.347505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b2a1a-ced5-4275-ac7a-85ab3ed676b0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.349107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ea1b2a1a-ced5-4275-ac7a-85ab3ed676b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.359947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] The token cache is being invalidate because project 24f9e146b8a84e679f9aa676584991f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:23.360258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.361689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.362978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b2a1a-ced5-4275-ac7a-85ab3ed676b0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.363771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 771/1535] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:23 2022] DELETE /identity/v3/projects/24f9e146b8a84e679f9aa676584991f4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:23.367512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.368861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.460506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67c891-7fca-4194-bbad-26ce3b0a2155 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.465507 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.466448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.466816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.467213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] PATH_INFO: `/v3/users/a9aaac532e324e5ca5bb856c8b3444d7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.468456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9aaac532e324e5ca5bb856c8b3444d7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.472058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.473458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.475501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.475935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.476519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.478930 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.498379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.500062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.522306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 765/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:23.533701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9aaac532e324e5ca5bb856c8b3444d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.534591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9aaac532e324e5ca5bb856c8b3444d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.553422 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9aaac532e324e5ca5bb856c8b3444d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.560742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9aaac532e324e5ca5bb856c8b3444d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.561535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-65701151-8c6a-4daa-8190-84b12c7119b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9aaac532e324e5ca5bb856c8b3444d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.575720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:23.585674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 772/1537] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:23 2022] DELETE /identity/v3/users/a9aaac532e324e5ca5bb856c8b3444d7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:23.939969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.956520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b2131-2122-41ca-ac56-57b74ae7afdc service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.036456 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.037219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.037494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.037871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] PATH_INFO: `/v3/projects/7ebaac6510dd45698698a4ac28a36f97` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.039257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ebaac6510dd45698698a4ac28a36f97)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.041371 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.118232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ebaac6510dd45698698a4ac28a36f97', 'request_id': 'req-d67d6ece-632f-487a-b235-77f62cf0992c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.127310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ebaac6510dd45698698a4ac28a36f97', 'request_id': 'req-d67d6ece-632f-487a-b235-77f62cf0992c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.146313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ebaac6510dd45698698a4ac28a36f97', 'request_id': 'req-d67d6ece-632f-487a-b235-77f62cf0992c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.148100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] The token cache is being invalidate because project 7ebaac6510dd45698698a4ac28a36f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:24.148100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.148100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d67d6ece-632f-487a-b235-77f62cf0992c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.149343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 766/1538] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:23 2022] DELETE /identity/v3/projects/7ebaac6510dd45698698a4ac28a36f97 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:24.155435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74059d-79ea-4a6e-b29c-b93c13a5b2ae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.156369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74059d-79ea-4a6e-b29c-b93c13a5b2ae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.156924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74059d-79ea-4a6e-b29c-b93c13a5b2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.179392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0c74059d-79ea-4a6e-b29c-b93c13a5b2ae None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:24.180999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c74059d-79ea-4a6e-b29c-b93c13a5b2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.289799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 773/1539] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:24.291432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b5d3-9430-48c5-b07d-6b6cbd5d60e2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.291741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b5d3-9430-48c5-b07d-6b6cbd5d60e2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.292044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b5d3-9430-48c5-b07d-6b6cbd5d60e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.332587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bcb6b5d3-9430-48c5-b07d-6b6cbd5d60e2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:24.334741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bcb6b5d3-9430-48c5-b07d-6b6cbd5d60e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.344969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 767/1540] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:24.349796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74059d-79ea-4a6e-b29c-b93c13a5b2ae None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.361557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74059d-79ea-4a6e-b29c-b93c13a5b2ae None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.371458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.372361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.372844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.373395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.374270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:24.375007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.377260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.409972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6b5d3-9430-48c5-b07d-6b6cbd5d60e2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.414768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:24.415587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 774/1541] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:24.420895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6b5d3-9430-48c5-b07d-6b6cbd5d60e2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.431244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9983c-8691-4a9f-968c-c23d7690b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.432071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9983c-8691-4a9f-968c-c23d7690b388 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.432532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9983c-8691-4a9f-968c-c23d7690b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.433183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9983c-8691-4a9f-968c-c23d7690b388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.434229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9983c-8691-4a9f-968c-c23d7690b388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.437344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9983c-8691-4a9f-968c-c23d7690b388 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.471730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 768/1542] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:24.476518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.487389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1618a2-1777-4859-9d82-e7f1b6c5ee89 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.549690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264e7c-006c-47bf-b4a5-34a872a80595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.550347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264e7c-006c-47bf-b4a5-34a872a80595 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.550620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264e7c-006c-47bf-b4a5-34a872a80595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.550886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264e7c-006c-47bf-b4a5-34a872a80595 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.551504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65264e7c-006c-47bf-b4a5-34a872a80595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.553413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65264e7c-006c-47bf-b4a5-34a872a80595 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.574048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 775/1543] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:24.578362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9983c-8691-4a9f-968c-c23d7690b388 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.589316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9983c-8691-4a9f-968c-c23d7690b388 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.595398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e612d-bbb6-4085-b210-657f57abbcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.596085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e612d-bbb6-4085-b210-657f57abbcf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.596360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e612d-bbb6-4085-b210-657f57abbcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.596628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e612d-bbb6-4085-b210-657f57abbcf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.597266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e612d-bbb6-4085-b210-657f57abbcf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.598908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e612d-bbb6-4085-b210-657f57abbcf1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.610257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 769/1544] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:24.615154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264e7c-006c-47bf-b4a5-34a872a80595 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.630078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264e7c-006c-47bf-b4a5-34a872a80595 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.639012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce84e9-eb18-42c0-8cfb-30ba62c34a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.639918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce84e9-eb18-42c0-8cfb-30ba62c34a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.640320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce84e9-eb18-42c0-8cfb-30ba62c34a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.640676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce84e9-eb18-42c0-8cfb-30ba62c34a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.641974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce84e9-eb18-42c0-8cfb-30ba62c34a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.643608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce84e9-eb18-42c0-8cfb-30ba62c34a37 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.655822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 776/1545] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:24.660328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e612d-bbb6-4085-b210-657f57abbcf1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.670424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e612d-bbb6-4085-b210-657f57abbcf1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.675054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.675747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.675998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.676268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] PATH_INFO: `/v3/projects/9698231c75fb47f693f7dcd0ced9e97d/users/e9a83e6dc5ed443f908da3a1ca8ecc87/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.691802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9698231c75fb47f693f7dcd0ced9e97d, user_id=e9a83e6dc5ed443f908da3a1ca8ecc87, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.694676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.709023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 770/1546] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:24 2022] PUT /identity/v3/projects/9698231c75fb47f693f7dcd0ced9e97d/users/e9a83e6dc5ed443f908da3a1ca8ecc87/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:24.720788 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc48d-68ad-4e42-9e7a-77c1aae66d10 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.721070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc48d-68ad-4e42-9e7a-77c1aae66d10 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.721342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc48d-68ad-4e42-9e7a-77c1aae66d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.749497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22dfc48d-68ad-4e42-9e7a-77c1aae66d10 None None] MFA Rules not processed for user `e9a83e6dc5ed443f908da3a1ca8ecc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:24.750998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22dfc48d-68ad-4e42-9e7a-77c1aae66d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.810622 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:24.825854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 777/1547] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:24.838454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.840349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.904501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3332ac6-a04e-4a22-996c-dbf271d69e68 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.908725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f83001-acc5-4bf9-9209-7bc4cf442ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.909354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f83001-acc5-4bf9-9209-7bc4cf442ecd service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.909632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f83001-acc5-4bf9-9209-7bc4cf442ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.909897 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f83001-acc5-4bf9-9209-7bc4cf442ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.919586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f83001-acc5-4bf9-9209-7bc4cf442ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.921026 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f83001-acc5-4bf9-9209-7bc4cf442ecd service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.930883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 771/1548] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:25.484346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfc48d-68ad-4e42-9e7a-77c1aae66d10 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.485960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22dfc48d-68ad-4e42-9e7a-77c1aae66d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:25.616546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfc48d-68ad-4e42-9e7a-77c1aae66d10 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.621925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.622115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.622300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.622551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] PATH_INFO: `/v3/users/1277eddad21746768f5ed0970ffdb0eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.623675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1277eddad21746768f5ed0970ffdb0eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.625640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.696491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1277eddad21746768f5ed0970ffdb0eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:25.697350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1277eddad21746768f5ed0970ffdb0eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:25.711878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1277eddad21746768f5ed0970ffdb0eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:25.726641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1277eddad21746768f5ed0970ffdb0eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:25.727394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1277eddad21746768f5ed0970ffdb0eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:25.744356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 778/1549] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:25 2022] DELETE /identity/v3/users/1277eddad21746768f5ed0970ffdb0eb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:25.749608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04696752-883b-472b-a729-11be3525c94b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.749608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04696752-883b-472b-a729-11be3525c94b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.749608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04696752-883b-472b-a729-11be3525c94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.771543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-04696752-883b-472b-a729-11be3525c94b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:25.772752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04696752-883b-472b-a729-11be3525c94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:25.842499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 772/1550] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:25.849922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.851466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:25.952231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee099b7-bf46-4d01-9d2d-fec41171e24d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.957352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f5a02-ab33-4c9d-97b5-149511398dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.958042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f5a02-ab33-4c9d-97b5-149511398dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.958426 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f5a02-ab33-4c9d-97b5-149511398dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.958768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f5a02-ab33-4c9d-97b5-149511398dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.970965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f5a02-ab33-4c9d-97b5-149511398dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.972682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f5a02-ab33-4c9d-97b5-149511398dd0 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.993584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 779/1551] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:26.086243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04696752-883b-472b-a729-11be3525c94b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:26.088198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04696752-883b-472b-a729-11be3525c94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:26.106111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04696752-883b-472b-a729-11be3525c94b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:26.111130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:26.111779 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.112062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.112364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] PATH_INFO: `/v3/users/f31d0ae8e82742deb83f850f8b55ae80` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.113887 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31d0ae8e82742deb83f850f8b55ae80)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.115880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.169176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:26.170868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31d0ae8e82742deb83f850f8b55ae80'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.171582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31d0ae8e82742deb83f850f8b55ae80'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.185693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31d0ae8e82742deb83f850f8b55ae80'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.193057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31d0ae8e82742deb83f850f8b55ae80'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.193953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31d0ae8e82742deb83f850f8b55ae80'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.207723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 773/1552] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:26 2022] DELETE /identity/v3/users/f31d0ae8e82742deb83f850f8b55ae80 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:26.212429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824606b-5b8c-48dd-9ebb-996bbb7fec86 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.212942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824606b-5b8c-48dd-9ebb-996bbb7fec86 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.213219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824606b-5b8c-48dd-9ebb-996bbb7fec86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.232792 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9824606b-5b8c-48dd-9ebb-996bbb7fec86 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:26.233961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9824606b-5b8c-48dd-9ebb-996bbb7fec86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:26.243681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:26.283387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:26.284941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:26.321006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 780/1553] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:26.326173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824606b-5b8c-48dd-9ebb-996bbb7fec86 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:26.330020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9824606b-5b8c-48dd-9ebb-996bbb7fec86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:26.387693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0cab8-dc36-48ac-b9e9-f6b1f059bf7c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:26.406958 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:26.407610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.407901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.408137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] PATH_INFO: `/v3/projects/8c7df5dfbc2b46c9b7492f9fa153b80e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.411551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7df5dfbc2b46c9b7492f9fa153b80e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.421533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.462866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824606b-5b8c-48dd-9ebb-996bbb7fec86 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:26.472724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399466c-b917-414a-904e-bcd3753515ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:26.473707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399466c-b917-414a-904e-bcd3753515ea service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.474336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399466c-b917-414a-904e-bcd3753515ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.474784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399466c-b917-414a-904e-bcd3753515ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.487007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399466c-b917-414a-904e-bcd3753515ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.489370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399466c-b917-414a-904e-bcd3753515ea service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.503053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7df5dfbc2b46c9b7492f9fa153b80e', 'request_id': 'req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.523771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7df5dfbc2b46c9b7492f9fa153b80e', 'request_id': 'req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.529074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 781/1554] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:26.539655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7df5dfbc2b46c9b7492f9fa153b80e', 'request_id': 'req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.593598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] The token cache is being invalidate because project 8c7df5dfbc2b46c9b7492f9fa153b80e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:26.594509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.595342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.603310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 774/1555] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:26 2022] DELETE /identity/v3/projects/8c7df5dfbc2b46c9b7492f9fa153b80e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:26.607782 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399466c-b917-414a-904e-bcd3753515ea service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:26.621946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399466c-b917-414a-904e-bcd3753515ea service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:26.739791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:26.740479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.740875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.741160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] PATH_INFO: `/v3/users/ed9f247b208a4ae59e0d9cc843cf4272` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.742532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed9f247b208a4ae59e0d9cc843cf4272)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.744658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.803509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9f247b208a4ae59e0d9cc843cf4272'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.804199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9f247b208a4ae59e0d9cc843cf4272'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.816382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:26.819417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9f247b208a4ae59e0d9cc843cf4272'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.829498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9f247b208a4ae59e0d9cc843cf4272'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.830432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9f247b208a4ae59e0d9cc843cf4272'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:26.845837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 782/1556] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:26 2022] DELETE /identity/v3/users/ed9f247b208a4ae59e0d9cc843cf4272 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:26.945484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:26.949159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:26.971580 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fa8bc-35de-4f1a-8e12-1a9cdf3195e4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.052833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:27.068517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.069409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.069682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.069950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] PATH_INFO: `/v3/projects/373652300a964d34a75002c71df7dd93` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.071984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=373652300a964d34a75002c71df7dd93)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.073953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.131578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '373652300a964d34a75002c71df7dd93', 'request_id': 'req-b264a9df-2280-4068-aa39-8cb4c401ac4a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.142580 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '373652300a964d34a75002c71df7dd93', 'request_id': 'req-b264a9df-2280-4068-aa39-8cb4c401ac4a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.150024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '373652300a964d34a75002c71df7dd93', 'request_id': 'req-b264a9df-2280-4068-aa39-8cb4c401ac4a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.162152 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] The token cache is being invalidate because project 373652300a964d34a75002c71df7dd93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:27.162446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.163107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.164612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 775/1557] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:26 2022] DELETE /identity/v3/projects/373652300a964d34a75002c71df7dd93 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:27.171378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.173792 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:27.181710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.184783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:27.207736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f18e3-528a-47ac-bd68-d3364c1e328d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.268110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264a9df-2280-4068-aa39-8cb4c401ac4a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.269801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.270469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.270806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.271072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] PATH_INFO: `/v3/users/5917cdfac9b242e2bd6bf3b59fa38cbf` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.272189 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5917cdfac9b242e2bd6bf3b59fa38cbf)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.273042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.273620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.273955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.274162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.274260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] PATH_INFO: `/v3/projects/5e186d48576e4d429cdd63b1fb07c186` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.275373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e186d48576e4d429cdd63b1fb07c186)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.277261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.323959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5917cdfac9b242e2bd6bf3b59fa38cbf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.324529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e186d48576e4d429cdd63b1fb07c186', 'request_id': 'req-41a1e2c6-facb-4976-953e-5bc57a84542f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.324629 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5917cdfac9b242e2bd6bf3b59fa38cbf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.336194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e186d48576e4d429cdd63b1fb07c186', 'request_id': 'req-41a1e2c6-facb-4976-953e-5bc57a84542f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.339161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5917cdfac9b242e2bd6bf3b59fa38cbf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.344118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e186d48576e4d429cdd63b1fb07c186', 'request_id': 'req-41a1e2c6-facb-4976-953e-5bc57a84542f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.350075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5917cdfac9b242e2bd6bf3b59fa38cbf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.350863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7a17b861-e591-4f21-9f1b-3d2c0670a930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5917cdfac9b242e2bd6bf3b59fa38cbf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.355298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] The token cache is being invalidate because project 5e186d48576e4d429cdd63b1fb07c186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:27.355582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.356237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41a1e2c6-facb-4976-953e-5bc57a84542f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.357739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 776/1558] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:27 2022] DELETE /identity/v3/projects/5e186d48576e4d429cdd63b1fb07c186 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:27.362894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20afab3-1a63-4d1f-9e91-5005038ffdfa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.363178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20afab3-1a63-4d1f-9e91-5005038ffdfa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.363448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20afab3-1a63-4d1f-9e91-5005038ffdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.365274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 783/1559] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:27 2022] DELETE /identity/v3/users/5917cdfac9b242e2bd6bf3b59fa38cbf => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:27.385617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d20afab3-1a63-4d1f-9e91-5005038ffdfa None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:27.386889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d20afab3-1a63-4d1f-9e91-5005038ffdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:27.396184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:27.447996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 777/1560] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:27.457210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126a8fb-2425-44fe-9096-8da997c4e4ca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.457495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126a8fb-2425-44fe-9096-8da997c4e4ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.457770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126a8fb-2425-44fe-9096-8da997c4e4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.478011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5126a8fb-2425-44fe-9096-8da997c4e4ca None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:27.479281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5126a8fb-2425-44fe-9096-8da997c4e4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:27.487214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 784/1561] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:27.491512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20afab3-1a63-4d1f-9e91-5005038ffdfa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.501799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20afab3-1a63-4d1f-9e91-5005038ffdfa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.506776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.507197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.507479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.507752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.508407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:27.508708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.510340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.522986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 778/1562] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:27.527680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126a8fb-2425-44fe-9096-8da997c4e4ca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.538954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126a8fb-2425-44fe-9096-8da997c4e4ca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.544159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.544777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.545054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.545309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.545944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.548549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.578465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 785/1563] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:27 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:27.582673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.592717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fd555-458c-473a-89f2-f6feef89bf3f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.653765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.656137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:27.668141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba859b-783f-4888-93f0-cbd4270cff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.668737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba859b-783f-4888-93f0-cbd4270cff2e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.669039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba859b-783f-4888-93f0-cbd4270cff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.669304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba859b-783f-4888-93f0-cbd4270cff2e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.669932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba859b-783f-4888-93f0-cbd4270cff2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.671998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba859b-783f-4888-93f0-cbd4270cff2e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.709836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 779/1564] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:27 2022] POST /identity/v3/users => generated 304 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:27.714047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba859b-783f-4888-93f0-cbd4270cff2e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.726803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba859b-783f-4888-93f0-cbd4270cff2e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.735947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:27.738706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cae0d-cc55-473e-b17d-5c822c217041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.739506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cae0d-cc55-473e-b17d-5c822c217041 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.739842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cae0d-cc55-473e-b17d-5c822c217041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.740123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cae0d-cc55-473e-b17d-5c822c217041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.740949 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cae0d-cc55-473e-b17d-5c822c217041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.743739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cae0d-cc55-473e-b17d-5c822c217041 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.762302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 780/1565] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:27.765596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cae0d-cc55-473e-b17d-5c822c217041 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.786561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fff74f-d35d-4f48-8864-8e6301fa0702 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.789522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cae0d-cc55-473e-b17d-5c822c217041 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.791869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.792501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.792775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.793044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] PATH_INFO: `/v3/projects/6ee4ff0a7f3844b68f1e946913ae1c1e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.794205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee4ff0a7f3844b68f1e946913ae1c1e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.795744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdadea-5af8-4f71-8565-762a8876ef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.796104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.798319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdadea-5af8-4f71-8565-762a8876ef19 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.799286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdadea-5af8-4f71-8565-762a8876ef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.800326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdadea-5af8-4f71-8565-762a8876ef19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.801132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdadea-5af8-4f71-8565-762a8876ef19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.803613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdadea-5af8-4f71-8565-762a8876ef19 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.816095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 781/1566] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:27.820418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdadea-5af8-4f71-8565-762a8876ef19 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.832414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdadea-5af8-4f71-8565-762a8876ef19 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.838777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a618d7-f0df-4aa2-8d48-18532d0ef498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.839708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a618d7-f0df-4aa2-8d48-18532d0ef498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.840176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a618d7-f0df-4aa2-8d48-18532d0ef498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.840499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a618d7-f0df-4aa2-8d48-18532d0ef498 admin admin] PATH_INFO: `/v3/projects/7aee503d682b4c19b47c632d890908e6/users/6579c55c435a46c3bea93f73e76c6a15/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.858054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a618d7-f0df-4aa2-8d48-18532d0ef498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aee503d682b4c19b47c632d890908e6, user_id=6579c55c435a46c3bea93f73e76c6a15, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.861306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a618d7-f0df-4aa2-8d48-18532d0ef498 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.866485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee4ff0a7f3844b68f1e946913ae1c1e', 'request_id': 'req-91589051-654c-42db-9536-6df97a60db9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.874661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee4ff0a7f3844b68f1e946913ae1c1e', 'request_id': 'req-91589051-654c-42db-9536-6df97a60db9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.879056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 782/1567] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:27 2022] PUT /identity/v3/projects/7aee503d682b4c19b47c632d890908e6/users/6579c55c435a46c3bea93f73e76c6a15/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:27.889655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691fcb1-3fae-40a6-9a50-a8571abcc9ef None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.890184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691fcb1-3fae-40a6-9a50-a8571abcc9ef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.890662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691fcb1-3fae-40a6-9a50-a8571abcc9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.902335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee4ff0a7f3844b68f1e946913ae1c1e', 'request_id': 'req-91589051-654c-42db-9536-6df97a60db9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.923254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f691fcb1-3fae-40a6-9a50-a8571abcc9ef None None] MFA Rules not processed for user `6579c55c435a46c3bea93f73e76c6a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:27.923603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] The token cache is being invalidate because project 6ee4ff0a7f3844b68f1e946913ae1c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:27.923899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.924437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f691fcb1-3fae-40a6-9a50-a8571abcc9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:27.924640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91589051-654c-42db-9536-6df97a60db9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:27.926373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 786/1568] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:27 2022] DELETE /identity/v3/projects/6ee4ff0a7f3844b68f1e946913ae1c1e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:27.930948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde22148-1550-44ea-ba0e-4a39b8d84880 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.931230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde22148-1550-44ea-ba0e-4a39b8d84880 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.931509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde22148-1550-44ea-ba0e-4a39b8d84880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.951745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fde22148-1550-44ea-ba0e-4a39b8d84880 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:27.952933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fde22148-1550-44ea-ba0e-4a39b8d84880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:28.040836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 787/1569] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:28.042523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f287b2-54c4-41cf-a7d8-3714bca13758 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.042818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f287b2-54c4-41cf-a7d8-3714bca13758 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.043099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f287b2-54c4-41cf-a7d8-3714bca13758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.064586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a1f287b2-54c4-41cf-a7d8-3714bca13758 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:28.065831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1f287b2-54c4-41cf-a7d8-3714bca13758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:28.073997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 788/1570] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:28.078352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f287b2-54c4-41cf-a7d8-3714bca13758 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.089388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f287b2-54c4-41cf-a7d8-3714bca13758 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.089940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:28.095545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.096145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.096420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.096689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.097333 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:28.097639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.099273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.111153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 789/1571] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:28.115587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.125138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6930ad6-972c-48ca-b417-9a8b78003cbf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.125300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 783/1572] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:27 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:28.129893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed321480-e660-473f-9277-3b5d0162f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.130259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691fcb1-3fae-40a6-9a50-a8571abcc9ef None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.130515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321480-e660-473f-9277-3b5d0162f33f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.130796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321480-e660-473f-9277-3b5d0162f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.131080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321480-e660-473f-9277-3b5d0162f33f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.131538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f691fcb1-3fae-40a6-9a50-a8571abcc9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:28.131755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed321480-e660-473f-9277-3b5d0162f33f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.133649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed321480-e660-473f-9277-3b5d0162f33f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.144291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691fcb1-3fae-40a6-9a50-a8571abcc9ef None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.149194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cd755-6cca-4ef2-94e5-ff297d45e7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.149821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cd755-6cca-4ef2-94e5-ff297d45e7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.150122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cd755-6cca-4ef2-94e5-ff297d45e7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.150379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cd755-6cca-4ef2-94e5-ff297d45e7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.151004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cd755-6cca-4ef2-94e5-ff297d45e7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.153392 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cd755-6cca-4ef2-94e5-ff297d45e7e3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.163052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 790/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:28.167279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed321480-e660-473f-9277-3b5d0162f33f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.185077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 784/1574] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/projects => generated 409 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:28.189293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cd755-6cca-4ef2-94e5-ff297d45e7e3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.193087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:28.239505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed321480-e660-473f-9277-3b5d0162f33f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.259375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cd755-6cca-4ef2-94e5-ff297d45e7e3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.264034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5bca9-a45d-467f-8ccf-1fe0a18a67a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.264649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5bca9-a45d-467f-8ccf-1fe0a18a67a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.266729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5bca9-a45d-467f-8ccf-1fe0a18a67a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.266729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5bca9-a45d-467f-8ccf-1fe0a18a67a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.266729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5bca9-a45d-467f-8ccf-1fe0a18a67a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.268049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5bca9-a45d-467f-8ccf-1fe0a18a67a6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.294506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 785/1575] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/users => generated 311 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:28.298007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5bca9-a45d-467f-8ccf-1fe0a18a67a6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.310227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2176c-c4ca-4a45-8fb2-f6ba8f4e805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.310720 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5bca9-a45d-467f-8ccf-1fe0a18a67a6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.310838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2176c-c4ca-4a45-8fb2-f6ba8f4e805e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.311105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2176c-c4ca-4a45-8fb2-f6ba8f4e805e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.311527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2176c-c4ca-4a45-8fb2-f6ba8f4e805e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.312185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2176c-c4ca-4a45-8fb2-f6ba8f4e805e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.315203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2176c-c4ca-4a45-8fb2-f6ba8f4e805e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.315945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b9ea-211d-466b-bb98-311f70a34364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.316563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b9ea-211d-466b-bb98-311f70a34364 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.316845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b9ea-211d-466b-bb98-311f70a34364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.317118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b9ea-211d-466b-bb98-311f70a34364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.317869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b9ea-211d-466b-bb98-311f70a34364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.319496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b9ea-211d-466b-bb98-311f70a34364 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.330928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 786/1576] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:28.335844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b9ea-211d-466b-bb98-311f70a34364 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.345723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 791/1577] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/users => generated 291 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:28.347245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b9ea-211d-466b-bb98-311f70a34364 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.347364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2176c-c4ca-4a45-8fb2-f6ba8f4e805e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.351479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4be7ce-8537-42f6-994b-2da7df7b0c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.352307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be7ce-8537-42f6-994b-2da7df7b0c28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.352743 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be7ce-8537-42f6-994b-2da7df7b0c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.353557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be7ce-8537-42f6-994b-2da7df7b0c28 admin admin] PATH_INFO: `/v3/projects/940655e9818c42178c82b44ee11b7587/users/88c7d185b11349a49447002ffb97a12f/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.362505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2176c-c4ca-4a45-8fb2-f6ba8f4e805e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.367711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f9c2e-cdc4-4986-934d-88115af64620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.368320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f9c2e-cdc4-4986-934d-88115af64620 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.368673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f9c2e-cdc4-4986-934d-88115af64620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.368857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f9c2e-cdc4-4986-934d-88115af64620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.369584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f9c2e-cdc4-4986-934d-88115af64620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.370917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4be7ce-8537-42f6-994b-2da7df7b0c28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940655e9818c42178c82b44ee11b7587, user_id=88c7d185b11349a49447002ffb97a12f, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.371134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f9c2e-cdc4-4986-934d-88115af64620 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.374129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4be7ce-8537-42f6-994b-2da7df7b0c28 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.383083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 792/1578] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:28.386854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f9c2e-cdc4-4986-934d-88115af64620 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.393027 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 787/1579] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:28 2022] PUT /identity/v3/projects/940655e9818c42178c82b44ee11b7587/users/88c7d185b11349a49447002ffb97a12f/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:28.396799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4be7ce-8537-42f6-994b-2da7df7b0c28 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.398249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f9c2e-cdc4-4986-934d-88115af64620 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.468051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7626b-114a-402d-a9d0-6feefc558d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.469052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7626b-114a-402d-a9d0-6feefc558d57 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.469430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7626b-114a-402d-a9d0-6feefc558d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.469768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7626b-114a-402d-a9d0-6feefc558d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.470502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7626b-114a-402d-a9d0-6feefc558d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.472162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7626b-114a-402d-a9d0-6feefc558d57 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.486578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 793/1580] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:28.490680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7626b-114a-402d-a9d0-6feefc558d57 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.492222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4be7ce-8537-42f6-994b-2da7df7b0c28 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.496864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecef0b-c2ff-4695-8ae1-e30b551ffe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.497472 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecef0b-c2ff-4695-8ae1-e30b551ffe03 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.497751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecef0b-c2ff-4695-8ae1-e30b551ffe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.498021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecef0b-c2ff-4695-8ae1-e30b551ffe03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.498700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecef0b-c2ff-4695-8ae1-e30b551ffe03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.500298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecef0b-c2ff-4695-8ae1-e30b551ffe03 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.500784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7626b-114a-402d-a9d0-6feefc558d57 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.505660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b18267-2e34-4533-a321-8ba9d2299c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.506838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18267-2e34-4533-a321-8ba9d2299c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.507113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18267-2e34-4533-a321-8ba9d2299c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.507377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18267-2e34-4533-a321-8ba9d2299c80 admin admin] PATH_INFO: `/v3/projects/32ac464245fc45a493daba57d57b432e/users/18433bafe1a14ef68184fe48b9e0c8f8/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.511045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 788/1581] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:28.515124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecef0b-c2ff-4695-8ae1-e30b551ffe03 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.520310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:28.523298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b18267-2e34-4533-a321-8ba9d2299c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ac464245fc45a493daba57d57b432e, user_id=18433bafe1a14ef68184fe48b9e0c8f8, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.525932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b18267-2e34-4533-a321-8ba9d2299c80 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.526194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecef0b-c2ff-4695-8ae1-e30b551ffe03 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.532089 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1684ff-5d14-41fe-b39c-72a14ec7c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.533045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1684ff-5d14-41fe-b39c-72a14ec7c3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.533303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1684ff-5d14-41fe-b39c-72a14ec7c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.533620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1684ff-5d14-41fe-b39c-72a14ec7c3bc admin admin] PATH_INFO: `/v3/projects/940655e9818c42178c82b44ee11b7587/users/88c7d185b11349a49447002ffb97a12f/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.535654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1684ff-5d14-41fe-b39c-72a14ec7c3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=940655e9818c42178c82b44ee11b7587, user_id=88c7d185b11349a49447002ffb97a12f, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.538479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1684ff-5d14-41fe-b39c-72a14ec7c3bc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.539302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 794/1582] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:28 2022] PUT /identity/v3/projects/32ac464245fc45a493daba57d57b432e/users/18433bafe1a14ef68184fe48b9e0c8f8/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:28.549761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc121ae-fc6e-4ef8-8e34-c40af5470cae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.549915 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 789/1583] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:28 2022] PUT /identity/v3/projects/940655e9818c42178c82b44ee11b7587/users/88c7d185b11349a49447002ffb97a12f/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:28.550269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc121ae-fc6e-4ef8-8e34-c40af5470cae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.550605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc121ae-fc6e-4ef8-8e34-c40af5470cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.560016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3b0e5-ff2d-455b-9924-bc1d8df86ffc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.560296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3b0e5-ff2d-455b-9924-bc1d8df86ffc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.560563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3b0e5-ff2d-455b-9924-bc1d8df86ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.579854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3fc121ae-fc6e-4ef8-8e34-c40af5470cae None None] MFA Rules not processed for user `18433bafe1a14ef68184fe48b9e0c8f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:28.581075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fc121ae-fc6e-4ef8-8e34-c40af5470cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:28.585066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:28.588627 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-93c3b0e5-ff2d-455b-9924-bc1d8df86ffc None None] MFA Rules not processed for user `88c7d185b11349a49447002ffb97a12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:28.589898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-93c3b0e5-ff2d-455b-9924-bc1d8df86ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:28.646481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 795/1584] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:28.651021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc121ae-fc6e-4ef8-8e34-c40af5470cae None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.659979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 790/1585] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:28.671369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3b0e5-ff2d-455b-9924-bc1d8df86ffc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.674262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-93c3b0e5-ff2d-455b-9924-bc1d8df86ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:28.716487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc121ae-fc6e-4ef8-8e34-c40af5470cae None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.721818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e959032-9aa0-41e1-8641-007295831aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.722457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959032-9aa0-41e1-8641-007295831aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.722737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959032-9aa0-41e1-8641-007295831aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.723010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959032-9aa0-41e1-8641-007295831aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.723642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e959032-9aa0-41e1-8641-007295831aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.725551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e959032-9aa0-41e1-8641-007295831aa2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.728535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3b0e5-ff2d-455b-9924-bc1d8df86ffc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.732037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478a1e6-d998-46bc-a280-65f1db3f77b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.732631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478a1e6-d998-46bc-a280-65f1db3f77b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.732906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478a1e6-d998-46bc-a280-65f1db3f77b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.733177 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478a1e6-d998-46bc-a280-65f1db3f77b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.742939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478a1e6-d998-46bc-a280-65f1db3f77b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.745122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478a1e6-d998-46bc-a280-65f1db3f77b9 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.756033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 791/1586] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:28.756174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 796/1587] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:28.759990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478a1e6-d998-46bc-a280-65f1db3f77b9 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.770334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478a1e6-d998-46bc-a280-65f1db3f77b9 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.830642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e44eb-4640-41f3-8466-db24f84d9c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.831241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e44eb-4640-41f3-8466-db24f84d9c5d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.831516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e44eb-4640-41f3-8466-db24f84d9c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.831786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e44eb-4640-41f3-8466-db24f84d9c5d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.832406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e44eb-4640-41f3-8466-db24f84d9c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.834315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e44eb-4640-41f3-8466-db24f84d9c5d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.857342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 792/1588] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:28 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:28.861723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e959032-9aa0-41e1-8641-007295831aa2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.872329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e959032-9aa0-41e1-8641-007295831aa2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.877216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.877816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.878123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.878470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.879112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.880744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.891011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 797/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:28.895388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e44eb-4640-41f3-8466-db24f84d9c5d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.905776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e44eb-4640-41f3-8466-db24f84d9c5d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.911105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293cf83-bd74-4ac1-8c18-2a1eae1fd6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.911862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293cf83-bd74-4ac1-8c18-2a1eae1fd6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.912218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293cf83-bd74-4ac1-8c18-2a1eae1fd6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.912567 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293cf83-bd74-4ac1-8c18-2a1eae1fd6fe admin admin] PATH_INFO: `/v3/projects/8456858d375c4a5aa77ade92c56a1f93/users/db2a3ae29c7e462fb80d44eaf72b5bfa/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.915228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:28.942584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293cf83-bd74-4ac1-8c18-2a1eae1fd6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8456858d375c4a5aa77ade92c56a1f93, user_id=db2a3ae29c7e462fb80d44eaf72b5bfa, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.945455 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293cf83-bd74-4ac1-8c18-2a1eae1fd6fe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.964508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 793/1590] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:28 2022] PUT /identity/v3/projects/8456858d375c4a5aa77ade92c56a1f93/users/db2a3ae29c7e462fb80d44eaf72b5bfa/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:28.967349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.969139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293cf83-bd74-4ac1-8c18-2a1eae1fd6fe admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.971824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:29.035112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49622a-9fba-4068-a34d-b672dd889a94 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:29.048444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293cf83-bd74-4ac1-8c18-2a1eae1fd6fe admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:29.054131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5381b-0741-42fe-b4aa-4a5754d6ae4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:29.054982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5381b-0741-42fe-b4aa-4a5754d6ae4c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.055364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5381b-0741-42fe-b4aa-4a5754d6ae4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.055737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5381b-0741-42fe-b4aa-4a5754d6ae4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:29.056538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5381b-0741-42fe-b4aa-4a5754d6ae4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:29.058328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5381b-0741-42fe-b4aa-4a5754d6ae4c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:29.071307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 794/1591] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:29.075686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5381b-0741-42fe-b4aa-4a5754d6ae4c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:29.085177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:29.088280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5381b-0741-42fe-b4aa-4a5754d6ae4c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:29.095371 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3b0e5-948e-4620-8ae3-777f169db52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:29.096161 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b0e5-948e-4620-8ae3-777f169db52c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.096640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b0e5-948e-4620-8ae3-777f169db52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.098192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b0e5-948e-4620-8ae3-777f169db52c admin admin] PATH_INFO: `/v3/projects/8456858d375c4a5aa77ade92c56a1f93/users/db2a3ae29c7e462fb80d44eaf72b5bfa/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:29.101523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b0e5-948e-4620-8ae3-777f169db52c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8456858d375c4a5aa77ade92c56a1f93, user_id=db2a3ae29c7e462fb80d44eaf72b5bfa, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:29.106118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b0e5-948e-4620-8ae3-777f169db52c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:29.112108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cda7c-d094-4095-81c9-d691d56ba08e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:29.113084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cda7c-d094-4095-81c9-d691d56ba08e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.113605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cda7c-d094-4095-81c9-d691d56ba08e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.113981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cda7c-d094-4095-81c9-d691d56ba08e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:29.121912 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 795/1592] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:29 2022] PUT /identity/v3/projects/8456858d375c4a5aa77ade92c56a1f93/users/db2a3ae29c7e462fb80d44eaf72b5bfa/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:29.134145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa8e61-e301-4934-91f9-8e74ceb423ec None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.134524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa8e61-e301-4934-91f9-8e74ceb423ec None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.134865 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa8e61-e301-4934-91f9-8e74ceb423ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:29.180302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-84aa8e61-e301-4934-91f9-8e74ceb423ec None None] MFA Rules not processed for user `db2a3ae29c7e462fb80d44eaf72b5bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:29.182478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-84aa8e61-e301-4934-91f9-8e74ceb423ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:29.215480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cda7c-d094-4095-81c9-d691d56ba08e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:29.216926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cda7c-d094-4095-81c9-d691d56ba08e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:29.227553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 798/1593] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:29.270814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 796/1594] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:29.285082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cda7c-d094-4095-81c9-d691d56ba08e service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:29.294518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cda7c-d094-4095-81c9-d691d56ba08e service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:29.348610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8929f95-e61d-4b38-8862-53437c9b0e4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:29.349787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929f95-e61d-4b38-8862-53437c9b0e4d service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.350206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929f95-e61d-4b38-8862-53437c9b0e4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.350613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8929f95-e61d-4b38-8862-53437c9b0e4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:29.363101 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8929f95-e61d-4b38-8862-53437c9b0e4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:29.365645 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8929f95-e61d-4b38-8862-53437c9b0e4d service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:29.376522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 799/1595] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:29.766331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa8e61-e301-4934-91f9-8e74ceb423ec None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:29.776036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa8e61-e301-4934-91f9-8e74ceb423ec None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:29.781747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db2a3ae29c7e462fb80d44eaf72b5bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8456858d375c4a5aa77ade92c56a1f93', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:29.782377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.782654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.782924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:29.783570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:29.783874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:29.785465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:29.796717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 797/1596] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:29.800868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8929f95-e61d-4b38-8862-53437c9b0e4d service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:29.809850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8929f95-e61d-4b38-8862-53437c9b0e4d service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:29.815008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35772bbc-18ea-46b5-8c19-505d8296ff80 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db2a3ae29c7e462fb80d44eaf72b5bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8456858d375c4a5aa77ade92c56a1f93', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:29.815606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35772bbc-18ea-46b5-8c19-505d8296ff80 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.815881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35772bbc-18ea-46b5-8c19-505d8296ff80 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.816146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35772bbc-18ea-46b5-8c19-505d8296ff80 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:29.816781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35772bbc-18ea-46b5-8c19-505d8296ff80 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:29.818751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35772bbc-18ea-46b5-8c19-505d8296ff80 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:29.848797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 800/1597] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:29 2022] POST /identity/v3/projects => generated 353 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:29.852929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:29.863309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c297f6-b4dd-4eaf-84c2-5b5e35cf5c2c tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:29.896147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:29.920729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db2a3ae29c7e462fb80d44eaf72b5bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8456858d375c4a5aa77ade92c56a1f93', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:29.921347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.921662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.921938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:29.922710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:29.923019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:29.924633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:29.936906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 798/1598] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:29.977667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35772bbc-18ea-46b5-8c19-505d8296ff80 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:29.989465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35772bbc-18ea-46b5-8c19-505d8296ff80 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:29.997777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db2a3ae29c7e462fb80d44eaf72b5bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8456858d375c4a5aa77ade92c56a1f93', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:29.998746 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:29.999116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:29.999480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] PATH_INFO: `/v3/projects/4631ec7aa1864af08c0469b01da7863c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:30.001297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4631ec7aa1864af08c0469b01da7863c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:30.004247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:30.068078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4631ec7aa1864af08c0469b01da7863c', 'request_id': 'req-813d38cf-78a5-481d-bc22-e2ebcd9497f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.075346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4631ec7aa1864af08c0469b01da7863c', 'request_id': 'req-813d38cf-78a5-481d-bc22-e2ebcd9497f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.084710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4631ec7aa1864af08c0469b01da7863c', 'request_id': 'req-813d38cf-78a5-481d-bc22-e2ebcd9497f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.088505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:30.097866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] The token cache is being invalidate because project 4631ec7aa1864af08c0469b01da7863c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:30.098512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.099449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813d38cf-78a5-481d-bc22-e2ebcd9497f1 tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.101038 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 801/1599] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:29 2022] DELETE /identity/v3/projects/4631ec7aa1864af08c0469b01da7863c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:30.300131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:30.302549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:30.372905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f604a70-ca71-414d-ab48-7903c8ccba3f tempest-VolumeQuotaClassesTest-730874382 tempest-VolumeQuotaClassesTest-730874382-project-admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:30.431106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:30.431809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:30.432215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:30.432610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] PATH_INFO: `/v3/users/18433bafe1a14ef68184fe48b9e0c8f8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:30.433951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18433bafe1a14ef68184fe48b9e0c8f8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:30.435884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:30.504785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18433bafe1a14ef68184fe48b9e0c8f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.505438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18433bafe1a14ef68184fe48b9e0c8f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.517470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18433bafe1a14ef68184fe48b9e0c8f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.524139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18433bafe1a14ef68184fe48b9e0c8f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.524797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18433bafe1a14ef68184fe48b9e0c8f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:30.535757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:30.542034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 799/1600] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:30 2022] DELETE /identity/v3/users/18433bafe1a14ef68184fe48b9e0c8f8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:30.546774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7225d31-d4a7-4e92-b9b7-ef1cff99e5f8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:30.547062 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7225d31-d4a7-4e92-b9b7-ef1cff99e5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:30.547410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7225d31-d4a7-4e92-b9b7-ef1cff99e5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:30.568356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d7225d31-d4a7-4e92-b9b7-ef1cff99e5f8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:30.569657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d7225d31-d4a7-4e92-b9b7-ef1cff99e5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:30.634056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 802/1601] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:30.644156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:30.645771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:30.710527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb12ff-d283-41d5-ab1b-568f6fe2a66f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:30.714562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e69408-21e1-42fa-8e3d-cf4a37d1bc1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:30.715180 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e69408-21e1-42fa-8e3d-cf4a37d1bc1a service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:30.715465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e69408-21e1-42fa-8e3d-cf4a37d1bc1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:30.715745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e69408-21e1-42fa-8e3d-cf4a37d1bc1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:30.725586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e69408-21e1-42fa-8e3d-cf4a37d1bc1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:30.727031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e69408-21e1-42fa-8e3d-cf4a37d1bc1a service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:30.736572 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 800/1602] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:31.105819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7225d31-d4a7-4e92-b9b7-ef1cff99e5f8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:31.107528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d7225d31-d4a7-4e92-b9b7-ef1cff99e5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:31.120902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7225d31-d4a7-4e92-b9b7-ef1cff99e5f8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:31.126314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:31.126979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:31.127227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:31.127491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] PATH_INFO: `/v3/projects/32ac464245fc45a493daba57d57b432e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:31.128774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ac464245fc45a493daba57d57b432e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:31.130731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:31.177148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ac464245fc45a493daba57d57b432e', 'request_id': 'req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.183103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ac464245fc45a493daba57d57b432e', 'request_id': 'req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.192314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ac464245fc45a493daba57d57b432e', 'request_id': 'req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.202961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] The token cache is being invalidate because project 32ac464245fc45a493daba57d57b432e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:31.203248 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.203905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.205408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 803/1603] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:31 2022] DELETE /identity/v3/projects/32ac464245fc45a493daba57d57b432e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:31.209473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e69408-21e1-42fa-8e3d-cf4a37d1bc1a service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:31.222895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e69408-21e1-42fa-8e3d-cf4a37d1bc1a service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:31.283010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:31.283651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:31.283941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:31.284218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] PATH_INFO: `/v3/users/db2a3ae29c7e462fb80d44eaf72b5bfa` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:31.285404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2a3ae29c7e462fb80d44eaf72b5bfa)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:31.287363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:31.338058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2a3ae29c7e462fb80d44eaf72b5bfa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.338749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2a3ae29c7e462fb80d44eaf72b5bfa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.351544 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2a3ae29c7e462fb80d44eaf72b5bfa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.358316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2a3ae29c7e462fb80d44eaf72b5bfa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.359003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-daf971f0-6b7d-446a-80f0-e046b38d621b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2a3ae29c7e462fb80d44eaf72b5bfa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.379639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 801/1604] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:31 2022] DELETE /identity/v3/users/db2a3ae29c7e462fb80d44eaf72b5bfa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:31.708220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:31.709642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:31.724639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9d79-bfb2-4614-bb45-2e7d5eddb96d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:31.820968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:31.821877 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:31.822345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:31.822561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] PATH_INFO: `/v3/projects/8456858d375c4a5aa77ade92c56a1f93` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:31.823713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=8456858d375c4a5aa77ade92c56a1f93)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:31.825697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:31.851739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:31.895925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8456858d375c4a5aa77ade92c56a1f93', 'request_id': 'req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.905389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8456858d375c4a5aa77ade92c56a1f93', 'request_id': 'req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.926974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8456858d375c4a5aa77ade92c56a1f93', 'request_id': 'req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.934778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] The token cache is being invalidate because project 8456858d375c4a5aa77ade92c56a1f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:31.935071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.936684 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19f4dca3-551d-4d47-a8a4-2b795a95c1ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:31.939624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 804/1605] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:31 2022] DELETE /identity/v3/projects/8456858d375c4a5aa77ade92c56a1f93 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:31.945552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941950a-2161-4b6f-88ba-69d7a9dbfe18 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:31.945839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941950a-2161-4b6f-88ba-69d7a9dbfe18 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:31.946128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941950a-2161-4b6f-88ba-69d7a9dbfe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:31.989189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2941950a-2161-4b6f-88ba-69d7a9dbfe18 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:31.990454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2941950a-2161-4b6f-88ba-69d7a9dbfe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:32.006222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:32.097915 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 802/1606] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:32.107201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ec1a9-2717-4b42-9888-1dcbbb9b3881 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.107739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ec1a9-2717-4b42-9888-1dcbbb9b3881 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.107811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ec1a9-2717-4b42-9888-1dcbbb9b3881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.145363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-816ec1a9-2717-4b42-9888-1dcbbb9b3881 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:32.146633 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-816ec1a9-2717-4b42-9888-1dcbbb9b3881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:32.166947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 805/1607] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:32.172241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941950a-2161-4b6f-88ba-69d7a9dbfe18 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.192391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941950a-2161-4b6f-88ba-69d7a9dbfe18 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:32.197667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:32.198493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.198969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.199301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.200149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:32.200558 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:32.202339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:32.217717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 803/1608] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:32.223089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ec1a9-2717-4b42-9888-1dcbbb9b3881 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.234573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ec1a9-2717-4b42-9888-1dcbbb9b3881 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:32.240748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e11d40-89eb-4f16-8636-af8e4a679224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:32.243095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e11d40-89eb-4f16-8636-af8e4a679224 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.243583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e11d40-89eb-4f16-8636-af8e4a679224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.243841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e11d40-89eb-4f16-8636-af8e4a679224 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.244505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e11d40-89eb-4f16-8636-af8e4a679224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:32.246573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e11d40-89eb-4f16-8636-af8e4a679224 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:32.284429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 806/1609] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:32 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:32.288475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.308597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd22593-fe71-4603-9a00-4d143949a4d3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:32.392064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b9908-007f-4512-aec5-5ff2f380b2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:32.392944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b9908-007f-4512-aec5-5ff2f380b2fb admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.393182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b9908-007f-4512-aec5-5ff2f380b2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.393563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b9908-007f-4512-aec5-5ff2f380b2fb admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.394723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b9908-007f-4512-aec5-5ff2f380b2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:32.397087 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b9908-007f-4512-aec5-5ff2f380b2fb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:32.437979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 804/1610] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:32 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:32.439585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e11d40-89eb-4f16-8636-af8e4a679224 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.451842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e11d40-89eb-4f16-8636-af8e4a679224 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:32.458557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a05282-2771-4d92-96f0-dde2b16663df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:32.462354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05282-2771-4d92-96f0-dde2b16663df admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.462354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05282-2771-4d92-96f0-dde2b16663df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.462354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05282-2771-4d92-96f0-dde2b16663df admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.462354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a05282-2771-4d92-96f0-dde2b16663df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:32.462739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a05282-2771-4d92-96f0-dde2b16663df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:32.477779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 807/1611] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:32.479985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b9908-007f-4512-aec5-5ff2f380b2fb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.498763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b9908-007f-4512-aec5-5ff2f380b2fb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:32.505547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65152fe6-10f6-4fd6-9d56-314f4b55b52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:32.506428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65152fe6-10f6-4fd6-9d56-314f4b55b52f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.506765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65152fe6-10f6-4fd6-9d56-314f4b55b52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.507457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65152fe6-10f6-4fd6-9d56-314f4b55b52f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.508906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65152fe6-10f6-4fd6-9d56-314f4b55b52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:32.518264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65152fe6-10f6-4fd6-9d56-314f4b55b52f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:32.530759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 805/1612] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:32.536044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a05282-2771-4d92-96f0-dde2b16663df admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.549468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a05282-2771-4d92-96f0-dde2b16663df admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:32.561715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:32.565314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.565314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.565314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] PATH_INFO: `/v3/projects/f2fd9c8b8dda457088e3fcdbad730805/users/5b4a0ba8945f4f569c994d992844be28/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.603351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fd9c8b8dda457088e3fcdbad730805, user_id=5b4a0ba8945f4f569c994d992844be28, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:32.605998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:32.624528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 808/1613] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:32 2022] PUT /identity/v3/projects/f2fd9c8b8dda457088e3fcdbad730805/users/5b4a0ba8945f4f569c994d992844be28/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:32.638052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18424e61-6356-4a25-a7fe-681963b0f547 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.638653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18424e61-6356-4a25-a7fe-681963b0f547 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.639085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18424e61-6356-4a25-a7fe-681963b0f547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.672224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-18424e61-6356-4a25-a7fe-681963b0f547 None None] MFA Rules not processed for user `5b4a0ba8945f4f569c994d992844be28`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:32.673433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-18424e61-6356-4a25-a7fe-681963b0f547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:32.755716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 806/1614] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:32.763050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.764746 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:32.810229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18424e61-6356-4a25-a7fe-681963b0f547 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.812272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-18424e61-6356-4a25-a7fe-681963b0f547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:32.838847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7a4f5-106c-4c0b-a316-aeb871e3e271 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:32.843513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420254b0-eafe-4a3f-8088-e28e06a1b9ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:32.844709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420254b0-eafe-4a3f-8088-e28e06a1b9ab service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.845096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420254b0-eafe-4a3f-8088-e28e06a1b9ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.845606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420254b0-eafe-4a3f-8088-e28e06a1b9ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.848077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:32.858304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420254b0-eafe-4a3f-8088-e28e06a1b9ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:32.860031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420254b0-eafe-4a3f-8088-e28e06a1b9ab service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:32.872466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 809/1615] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:32.903231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420254b0-eafe-4a3f-8088-e28e06a1b9ab service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:32.925298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18424e61-6356-4a25-a7fe-681963b0f547 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:32.930308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:32.931561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.931720 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.932216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] PATH_INFO: `/v3/users/edb280de960b4682a1ff2b36956c8cc0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.933549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb280de960b4682a1ff2b36956c8cc0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:32.935572 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:32.964965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:33.003482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420254b0-eafe-4a3f-8088-e28e06a1b9ab service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.008658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb280de960b4682a1ff2b36956c8cc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.008895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.009425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb280de960b4682a1ff2b36956c8cc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.009691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.010034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.010411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] PATH_INFO: `/v3/users/5b4a0ba8945f4f569c994d992844be28` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.011649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4a0ba8945f4f569c994d992844be28)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.014650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.029325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb280de960b4682a1ff2b36956c8cc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.038767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb280de960b4682a1ff2b36956c8cc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.039822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c0c012f4-8ad5-4599-8cd1-a3986dd727ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb280de960b4682a1ff2b36956c8cc0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.054477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 807/1616] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:32 2022] DELETE /identity/v3/users/edb280de960b4682a1ff2b36956c8cc0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:33.059331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992588c-6250-4e8f-8c13-dfdc7eaf4ced None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.059855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992588c-6250-4e8f-8c13-dfdc7eaf4ced None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.060194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992588c-6250-4e8f-8c13-dfdc7eaf4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.072778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4a0ba8945f4f569c994d992844be28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.073505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4a0ba8945f4f569c994d992844be28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.089632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4a0ba8945f4f569c994d992844be28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.096941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4a0ba8945f4f569c994d992844be28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.097431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b992588c-6250-4e8f-8c13-dfdc7eaf4ced None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:33.097771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b905d512-9fbe-4a0b-8299-38f59da53a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4a0ba8945f4f569c994d992844be28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.098840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b992588c-6250-4e8f-8c13-dfdc7eaf4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.111687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 810/1617] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:32 2022] DELETE /identity/v3/users/5b4a0ba8945f4f569c994d992844be28 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:33.116227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3164429-4725-471f-b257-56f569752f66 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.116512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3164429-4725-471f-b257-56f569752f66 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.116790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3164429-4725-471f-b257-56f569752f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.135099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f3164429-4725-471f-b257-56f569752f66 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:33.136292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f3164429-4725-471f-b257-56f569752f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.160365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 808/1618] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:33.170236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992588c-6250-4e8f-8c13-dfdc7eaf4ced None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.171596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b992588c-6250-4e8f-8c13-dfdc7eaf4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.195697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 811/1619] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:33.205722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3164429-4725-471f-b257-56f569752f66 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.207886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f3164429-4725-471f-b257-56f569752f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.211353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:33.231496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992588c-6250-4e8f-8c13-dfdc7eaf4ced None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.266158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3164429-4725-471f-b257-56f569752f66 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.269810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.270446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.270729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.271002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.280292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.280721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.281363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.281654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.281968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.282076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.291060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.292416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 812/1620] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:33.292535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.302297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 809/1621] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:33.715346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.716913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.732233 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29535-832e-4b50-9ad9-384b667ef98d service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.736474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.738259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.740214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.740844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.741127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.741390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] PATH_INFO: `/v3/projects/d04ac24612bd4188ada9425297e87193` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.742709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d04ac24612bd4188ada9425297e87193)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.744541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.747307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:33.784954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492e670-834c-4bc7-82af-2aae5724b030 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.805624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd04ac24612bd4188ada9425297e87193', 'request_id': 'req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.807068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.807734 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.808017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.808292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] PATH_INFO: `/v3/projects/f2fd9c8b8dda457088e3fcdbad730805` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.813183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fd9c8b8dda457088e3fcdbad730805)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.814125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd04ac24612bd4188ada9425297e87193', 'request_id': 'req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.815208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.824696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd04ac24612bd4188ada9425297e87193', 'request_id': 'req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.839517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] The token cache is being invalidate because project d04ac24612bd4188ada9425297e87193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:33.840389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.840672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.842424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 813/1622] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:33 2022] DELETE /identity/v3/projects/d04ac24612bd4188ada9425297e87193 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:33.847752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.849170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.907144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fd9c8b8dda457088e3fcdbad730805', 'request_id': 'req-4087e493-442c-4180-bfd9-f2069bffae97'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.915808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fd9c8b8dda457088e3fcdbad730805', 'request_id': 'req-4087e493-442c-4180-bfd9-f2069bffae97'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.923206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4712d1-aae7-4859-a931-1d1fa4ad4f9d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.923354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fd9c8b8dda457088e3fcdbad730805', 'request_id': 'req-4087e493-442c-4180-bfd9-f2069bffae97'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.933962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] The token cache is being invalidate because project f2fd9c8b8dda457088e3fcdbad730805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:33.934408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.938504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4087e493-442c-4180-bfd9-f2069bffae97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.940470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 810/1623] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:33 2022] DELETE /identity/v3/projects/f2fd9c8b8dda457088e3fcdbad730805 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:33.944938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80832-c6a8-472f-9d7e-e407c9cd5d88 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.945243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80832-c6a8-472f-9d7e-e407c9cd5d88 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.945516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80832-c6a8-472f-9d7e-e407c9cd5d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.967905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bbe80832-c6a8-472f-9d7e-e407c9cd5d88 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:33.969754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bbe80832-c6a8-472f-9d7e-e407c9cd5d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.983444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.984080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.984352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.984615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] PATH_INFO: `/v3/users/e7a626311e754eb4a0d7dfe7f93ad629` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.985763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a626311e754eb4a0d7dfe7f93ad629)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.987712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.033707 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 811/1624] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:34.042294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c777522-5249-4464-9492-4a37388184aa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.042571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c777522-5249-4464-9492-4a37388184aa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.042839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c777522-5249-4464-9492-4a37388184aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.046246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:34.055707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a626311e754eb4a0d7dfe7f93ad629'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.056592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a626311e754eb4a0d7dfe7f93ad629'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.063440 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7c777522-5249-4464-9492-4a37388184aa None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:34.064742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7c777522-5249-4464-9492-4a37388184aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.069266 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a626311e754eb4a0d7dfe7f93ad629'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.072860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 812/1625] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:34.077201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c777522-5249-4464-9492-4a37388184aa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.077411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a626311e754eb4a0d7dfe7f93ad629'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.078131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a626311e754eb4a0d7dfe7f93ad629'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.086536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c777522-5249-4464-9492-4a37388184aa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.091816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.092311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.092589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.092875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.093525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:34.094239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.095837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.100315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 814/1626] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:33 2022] DELETE /identity/v3/users/e7a626311e754eb4a0d7dfe7f93ad629 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:34.106472 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:34.114276 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 813/1627] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:34.119369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.189834 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970e90f-8e33-4164-a8ed-9676154fb9ec admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.194406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec70cd-d56c-4073-8545-d8ad476464ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.195009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec70cd-d56c-4073-8545-d8ad476464ff admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.195284 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec70cd-d56c-4073-8545-d8ad476464ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.195547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec70cd-d56c-4073-8545-d8ad476464ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.196177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec70cd-d56c-4073-8545-d8ad476464ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.198799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec70cd-d56c-4073-8545-d8ad476464ff admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.229752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 815/1628] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/projects => generated 359 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:34.231815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.243229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e43b30-78cc-40f2-8427-e3b385a151a7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.303667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0672c8-58a9-4b2e-b89c-bb426806a16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.304305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0672c8-58a9-4b2e-b89c-bb426806a16c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.304586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0672c8-58a9-4b2e-b89c-bb426806a16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.304854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0672c8-58a9-4b2e-b89c-bb426806a16c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.305602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0672c8-58a9-4b2e-b89c-bb426806a16c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.307713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0672c8-58a9-4b2e-b89c-bb426806a16c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.333638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 814/1629] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/users => generated 280 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:34.334932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec70cd-d56c-4073-8545-d8ad476464ff admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.345705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec70cd-d56c-4073-8545-d8ad476464ff admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.350767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50518c-4526-4e55-93bd-a06e154db060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.351372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50518c-4526-4e55-93bd-a06e154db060 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.351709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50518c-4526-4e55-93bd-a06e154db060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.351983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50518c-4526-4e55-93bd-a06e154db060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.352682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50518c-4526-4e55-93bd-a06e154db060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.354323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50518c-4526-4e55-93bd-a06e154db060 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.366203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 816/1630] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:34.370430 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0672c8-58a9-4b2e-b89c-bb426806a16c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.381232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0672c8-58a9-4b2e-b89c-bb426806a16c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.387053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56052e9f-34fe-4aed-88d5-09dc2e7ad900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.387323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052e9f-34fe-4aed-88d5-09dc2e7ad900 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.387668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052e9f-34fe-4aed-88d5-09dc2e7ad900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.387883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052e9f-34fe-4aed-88d5-09dc2e7ad900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.388501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56052e9f-34fe-4aed-88d5-09dc2e7ad900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.390128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56052e9f-34fe-4aed-88d5-09dc2e7ad900 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.401669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 815/1631] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:34.410322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50518c-4526-4e55-93bd-a06e154db060 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.426323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50518c-4526-4e55-93bd-a06e154db060 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.430769 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.431466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.431765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.432652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] PATH_INFO: `/v3/projects/497d5733f94b49839b875845304e1a8b/users/4cffb8da58294c9cbe4659ebcbc63e33/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.449160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=497d5733f94b49839b875845304e1a8b, user_id=4cffb8da58294c9cbe4659ebcbc63e33, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.452032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.466248 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 817/1632] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:34 2022] PUT /identity/v3/projects/497d5733f94b49839b875845304e1a8b/users/4cffb8da58294c9cbe4659ebcbc63e33/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:34.477311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed9164-e6ca-45e4-b8c1-4b76af8fc437 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.477670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed9164-e6ca-45e4-b8c1-4b76af8fc437 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.478038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed9164-e6ca-45e4-b8c1-4b76af8fc437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.507088 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-24ed9164-e6ca-45e4-b8c1-4b76af8fc437 None None] MFA Rules not processed for user `4cffb8da58294c9cbe4659ebcbc63e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:34.508388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-24ed9164-e6ca-45e4-b8c1-4b76af8fc437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.546600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.549694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.585106 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 816/1633] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:34.619658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617df878-900c-4fe2-832c-0376ce576d11 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.621058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed9164-e6ca-45e4-b8c1-4b76af8fc437 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.622467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-24ed9164-e6ca-45e4-b8c1-4b76af8fc437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.625159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.625895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.626282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.626552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] PATH_INFO: `/v3/projects/dea86ab8b12345428cd04828c1ea4be3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.627814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea86ab8b12345428cd04828c1ea4be3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.630363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.642550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:34.682398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea86ab8b12345428cd04828c1ea4be3', 'request_id': 'req-e092fc7d-93ce-434e-8438-8f813cd85647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.682725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed9164-e6ca-45e4-b8c1-4b76af8fc437 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.688406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea86ab8b12345428cd04828c1ea4be3', 'request_id': 'req-e092fc7d-93ce-434e-8438-8f813cd85647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.696448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea86ab8b12345428cd04828c1ea4be3', 'request_id': 'req-e092fc7d-93ce-434e-8438-8f813cd85647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.700988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:34.708113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] The token cache is being invalidate because project dea86ab8b12345428cd04828c1ea4be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:34.708113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.708918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e092fc7d-93ce-434e-8438-8f813cd85647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.710457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 818/1634] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:34 2022] DELETE /identity/v3/projects/dea86ab8b12345428cd04828c1ea4be3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:34.715139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb82d-ac50-49af-96f8-30f668ce308b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.715412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb82d-ac50-49af-96f8-30f668ce308b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.715681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb82d-ac50-49af-96f8-30f668ce308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.735966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2d0cb82d-ac50-49af-96f8-30f668ce308b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:34.736142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.736741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.737283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.737368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d0cb82d-ac50-49af-96f8-30f668ce308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.737450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.739439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.796630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.796960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 819/1635] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:34.799281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.806322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192ab03-b806-48d5-bfdc-64ac7fbb3b18 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.806538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192ab03-b806-48d5-bfdc-64ac7fbb3b18 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.806807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192ab03-b806-48d5-bfdc-64ac7fbb3b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.808656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 817/1636] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:34.824399 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.825819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.826449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d192ab03-b806-48d5-bfdc-64ac7fbb3b18 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:34.827646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d192ab03-b806-48d5-bfdc-64ac7fbb3b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.835895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 820/1637] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:34.839810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192ab03-b806-48d5-bfdc-64ac7fbb3b18 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.840126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a7169-5f22-49eb-a54c-e724f30f1eed service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.845119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.845739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.846019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.846321 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] PATH_INFO: `/v3/users/4cffb8da58294c9cbe4659ebcbc63e33` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.847439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cffb8da58294c9cbe4659ebcbc63e33)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.849335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.849878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192ab03-b806-48d5-bfdc-64ac7fbb3b18 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.854553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.855133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.855407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.855674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.856312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:34.856613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.858203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.869937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 821/1638] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:34.874320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.887798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb938-3bc8-4fb9-b619-7802002f4f40 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.892832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.893741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.894026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.894341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.894998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.896973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.915841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cffb8da58294c9cbe4659ebcbc63e33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.916811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cffb8da58294c9cbe4659ebcbc63e33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.929328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 822/1639] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:34.933642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.935605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.941313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cffb8da58294c9cbe4659ebcbc63e33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.956427 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cffb8da58294c9cbe4659ebcbc63e33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.957190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fabc0a87-8501-4f8d-96d9-f638b1412875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cffb8da58294c9cbe4659ebcbc63e33'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:34.978170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 818/1640] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:34 2022] DELETE /identity/v3/users/4cffb8da58294c9cbe4659ebcbc63e33 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:34.983211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d899-7839-432a-b2b6-4e9b2ff1b93e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.983754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d899-7839-432a-b2b6-4e9b2ff1b93e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.984559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d899-7839-432a-b2b6-4e9b2ff1b93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.014520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4920d899-7839-432a-b2b6-4e9b2ff1b93e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:35.015729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4920d899-7839-432a-b2b6-4e9b2ff1b93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.020342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:35.024439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02359d-cbaf-43e7-8e0d-dbf4a7c6ac3f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.073672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:35.097844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 819/1641] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:35.106902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b10a-4058-4e5a-b4b8-868653f94149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.107527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b10a-4058-4e5a-b4b8-868653f94149 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.107839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b10a-4058-4e5a-b4b8-868653f94149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.108115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b10a-4058-4e5a-b4b8-868653f94149 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.108760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324b10a-4058-4e5a-b4b8-868653f94149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.109193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920d899-7839-432a-b2b6-4e9b2ff1b93e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.110934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324b10a-4058-4e5a-b4b8-868653f94149 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.111057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4920d899-7839-432a-b2b6-4e9b2ff1b93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.136842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 823/1642] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/users => generated 300 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:35.140698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b10a-4058-4e5a-b4b8-868653f94149 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.153297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b10a-4058-4e5a-b4b8-868653f94149 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.158888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea815a-cfad-4b05-994e-7c342704f51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.159602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea815a-cfad-4b05-994e-7c342704f51e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.159954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea815a-cfad-4b05-994e-7c342704f51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.160296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea815a-cfad-4b05-994e-7c342704f51e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.161022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea815a-cfad-4b05-994e-7c342704f51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.162799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea815a-cfad-4b05-994e-7c342704f51e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.174018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 824/1643] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:35.178179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea815a-cfad-4b05-994e-7c342704f51e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.181589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920d899-7839-432a-b2b6-4e9b2ff1b93e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.185903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68a956-9f54-4552-bcf1-7830ebdb488f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.186782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68a956-9f54-4552-bcf1-7830ebdb488f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.187206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68a956-9f54-4552-bcf1-7830ebdb488f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.187621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68a956-9f54-4552-bcf1-7830ebdb488f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.188913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea815a-cfad-4b05-994e-7c342704f51e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.193604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f7dd-2c1f-4223-8452-e5f8e96fe1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.194253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f7dd-2c1f-4223-8452-e5f8e96fe1bd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.194535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f7dd-2c1f-4223-8452-e5f8e96fe1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.194809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f7dd-2c1f-4223-8452-e5f8e96fe1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.195471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674f7dd-2c1f-4223-8452-e5f8e96fe1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.197098 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674f7dd-2c1f-4223-8452-e5f8e96fe1bd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.199954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68a956-9f54-4552-bcf1-7830ebdb488f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.201636 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68a956-9f54-4552-bcf1-7830ebdb488f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.209211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 825/1644] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:35.215002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f7dd-2c1f-4223-8452-e5f8e96fe1bd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.216011 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 820/1645] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:35.226523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f7dd-2c1f-4223-8452-e5f8e96fe1bd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.231557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf0437-6213-4c6b-9bbc-5d05d8cc7dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.232340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf0437-6213-4c6b-9bbc-5d05d8cc7dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.232495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf0437-6213-4c6b-9bbc-5d05d8cc7dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.232923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf0437-6213-4c6b-9bbc-5d05d8cc7dcb admin admin] PATH_INFO: `/v3/projects/19831341772a4015927c029f947030aa/users/bf33e3330d5c4de1952c2878164090ed/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.253986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf0437-6213-4c6b-9bbc-5d05d8cc7dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=19831341772a4015927c029f947030aa, user_id=bf33e3330d5c4de1952c2878164090ed, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.257346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf0437-6213-4c6b-9bbc-5d05d8cc7dcb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.280218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 826/1646] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:35 2022] PUT /identity/v3/projects/19831341772a4015927c029f947030aa/users/bf33e3330d5c4de1952c2878164090ed/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:35.286349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091e97-747e-4b57-a306-ee7051a3394e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.286903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091e97-747e-4b57-a306-ee7051a3394e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.287439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091e97-747e-4b57-a306-ee7051a3394e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.319373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-00091e97-747e-4b57-a306-ee7051a3394e None None] MFA Rules not processed for user `bf33e3330d5c4de1952c2878164090ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:35.320496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-00091e97-747e-4b57-a306-ee7051a3394e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.423496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 821/1647] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:35.428172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf0437-6213-4c6b-9bbc-5d05d8cc7dcb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.496296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf0437-6213-4c6b-9bbc-5d05d8cc7dcb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.501240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3acf64-a1dd-4d01-87d2-31ce8c3d2292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.501854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3acf64-a1dd-4d01-87d2-31ce8c3d2292 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.502154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3acf64-a1dd-4d01-87d2-31ce8c3d2292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.502429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3acf64-a1dd-4d01-87d2-31ce8c3d2292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.503109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3acf64-a1dd-4d01-87d2-31ce8c3d2292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.505019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3acf64-a1dd-4d01-87d2-31ce8c3d2292 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.535093 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 827/1648] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:35 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:35.539470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091e97-747e-4b57-a306-ee7051a3394e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.550632 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091e97-747e-4b57-a306-ee7051a3394e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.615049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.616297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.616576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.616847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.617485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.619440 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.649134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 822/1649] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:35 2022] POST /identity/v3/users => generated 306 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:35.650872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3acf64-a1dd-4d01-87d2-31ce8c3d2292 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.662491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3acf64-a1dd-4d01-87d2-31ce8c3d2292 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.669986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e85b4-510f-4463-8efe-9d6ead402138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.670582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.670823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e85b4-510f-4463-8efe-9d6ead402138 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.671103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e85b4-510f-4463-8efe-9d6ead402138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.671372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e85b4-510f-4463-8efe-9d6ead402138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.672014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e85b4-510f-4463-8efe-9d6ead402138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.672867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.673878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e85b4-510f-4463-8efe-9d6ead402138 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.686005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3be8-dfe0-4677-abb6-cc6881e5804f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.689703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 828/1650] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:35.690986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e85b4-510f-4463-8efe-9d6ead402138 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.695473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.696309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.702145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.702465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] PATH_INFO: `/v3/projects/497d5733f94b49839b875845304e1a8b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.705092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=497d5733f94b49839b875845304e1a8b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.707243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.719473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e85b4-510f-4463-8efe-9d6ead402138 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.721513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:35.732853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccbf29-ec1f-4b15-aed7-47c5b6b3a34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.734018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbf29-ec1f-4b15-aed7-47c5b6b3a34c admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.734333 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbf29-ec1f-4b15-aed7-47c5b6b3a34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.734609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbf29-ec1f-4b15-aed7-47c5b6b3a34c admin admin] PATH_INFO: `/v3/projects/d2abcef2998f42e5a049e3c98aec6dcb/users/db5bf03605f74b9580711a746107db65/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.750114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccbf29-ec1f-4b15-aed7-47c5b6b3a34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2abcef2998f42e5a049e3c98aec6dcb, user_id=db5bf03605f74b9580711a746107db65, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.753505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccbf29-ec1f-4b15-aed7-47c5b6b3a34c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.762359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:35.768616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 829/1651] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:35 2022] PUT /identity/v3/projects/d2abcef2998f42e5a049e3c98aec6dcb/users/db5bf03605f74b9580711a746107db65/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:35.772562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccbf29-ec1f-4b15-aed7-47c5b6b3a34c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.779555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '497d5733f94b49839b875845304e1a8b', 'request_id': 'req-7a85aafb-b99c-43a3-925a-83c3259e89f3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.788530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '497d5733f94b49839b875845304e1a8b', 'request_id': 'req-7a85aafb-b99c-43a3-925a-83c3259e89f3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.796011 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '497d5733f94b49839b875845304e1a8b', 'request_id': 'req-7a85aafb-b99c-43a3-925a-83c3259e89f3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.807205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] The token cache is being invalidate because project 497d5733f94b49839b875845304e1a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:35.808391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.809109 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7a85aafb-b99c-43a3-925a-83c3259e89f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.810672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 823/1652] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:35 2022] DELETE /identity/v3/projects/497d5733f94b49839b875845304e1a8b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:35.815696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fd8d0-27cb-44cf-b566-9114f3185660 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.816020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fd8d0-27cb-44cf-b566-9114f3185660 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.816267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fd8d0-27cb-44cf-b566-9114f3185660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.835841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fc2fd8d0-27cb-44cf-b566-9114f3185660 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:35.837028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fc2fd8d0-27cb-44cf-b566-9114f3185660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.837540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccbf29-ec1f-4b15-aed7-47c5b6b3a34c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.894398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.896144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 824/1653] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:35.902697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.902697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.902697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.902697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.902697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.903934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53efcb-6053-4693-ab46-c4bacfca5db8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.904223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53efcb-6053-4693-ab46-c4bacfca5db8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.904497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53efcb-6053-4693-ab46-c4bacfca5db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.912749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 830/1654] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:35.916870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.918758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.930875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d2b97-9643-41ae-8459-2f75b5eafc11 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.931628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5d53efcb-6053-4693-ab46-c4bacfca5db8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:35.932812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5d53efcb-6053-4693-ab46-c4bacfca5db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.935373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ecaeb-b2e7-496a-96c2-dc50eae7adcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.936016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ecaeb-b2e7-496a-96c2-dc50eae7adcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.936294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ecaeb-b2e7-496a-96c2-dc50eae7adcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.936560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ecaeb-b2e7-496a-96c2-dc50eae7adcb admin admin] PATH_INFO: `/v3/projects/d2abcef2998f42e5a049e3c98aec6dcb/users/db5bf03605f74b9580711a746107db65/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.938840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ecaeb-b2e7-496a-96c2-dc50eae7adcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2abcef2998f42e5a049e3c98aec6dcb, user_id=db5bf03605f74b9580711a746107db65, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.941081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 825/1655] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:35.941465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ecaeb-b2e7-496a-96c2-dc50eae7adcb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.945431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53efcb-6053-4693-ab46-c4bacfca5db8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.951935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 831/1656] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:35 2022] PUT /identity/v3/projects/d2abcef2998f42e5a049e3c98aec6dcb/users/db5bf03605f74b9580711a746107db65/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:35.956954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53efcb-6053-4693-ab46-c4bacfca5db8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.965855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d3b2f-1416-44b3-bcd1-32cb6fff0c1b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.966261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d3b2f-1416-44b3-bcd1-32cb6fff0c1b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.966568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d3b2f-1416-44b3-bcd1-32cb6fff0c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.003117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8e0d3b2f-1416-44b3-bcd1-32cb6fff0c1b None None] MFA Rules not processed for user `db5bf03605f74b9580711a746107db65`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.004349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e0d3b2f-1416-44b3-bcd1-32cb6fff0c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.029906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.030624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.030900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.031243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.032078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:36.032240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.034050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.054243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 826/1657] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:36.055338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.065287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce784c3-7532-440e-9994-6c04c1bdd81c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.069688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c43f2f-7661-4c13-9c40-bd5d41892a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.070299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c43f2f-7661-4c13-9c40-bd5d41892a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.070571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c43f2f-7661-4c13-9c40-bd5d41892a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.072424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c43f2f-7661-4c13-9c40-bd5d41892a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.073060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c43f2f-7661-4c13-9c40-bd5d41892a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.075149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c43f2f-7661-4c13-9c40-bd5d41892a9a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.083367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 832/1658] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:35 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:36.095666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d3b2f-1416-44b3-bcd1-32cb6fff0c1b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.098512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e0d3b2f-1416-44b3-bcd1-32cb6fff0c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.101936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:36.113807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 827/1659] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:36.114978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c43f2f-7661-4c13-9c40-bd5d41892a9a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.120212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:36.166958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d3b2f-1416-44b3-bcd1-32cb6fff0c1b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.178677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c43f2f-7661-4c13-9c40-bd5d41892a9a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.183342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703f0f8-6dbb-4dbf-b249-0d2b70bd0b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.183960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703f0f8-6dbb-4dbf-b249-0d2b70bd0b07 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.184242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703f0f8-6dbb-4dbf-b249-0d2b70bd0b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.184516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703f0f8-6dbb-4dbf-b249-0d2b70bd0b07 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.185146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703f0f8-6dbb-4dbf-b249-0d2b70bd0b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.187101 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703f0f8-6dbb-4dbf-b249-0d2b70bd0b07 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.210191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 828/1660] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:36.213688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703f0f8-6dbb-4dbf-b249-0d2b70bd0b07 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.215853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c5a4c-175f-48f9-a573-2227c647a2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.216470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c5a4c-175f-48f9-a573-2227c647a2fc service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.216751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c5a4c-175f-48f9-a573-2227c647a2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.217026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c5a4c-175f-48f9-a573-2227c647a2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.224198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703f0f8-6dbb-4dbf-b249-0d2b70bd0b07 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.229748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15ade2-3cf3-4065-a6cf-42a6f1f8a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.230382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15ade2-3cf3-4065-a6cf-42a6f1f8a7db admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.230681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15ade2-3cf3-4065-a6cf-42a6f1f8a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.230948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15ade2-3cf3-4065-a6cf-42a6f1f8a7db admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.231592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15ade2-3cf3-4065-a6cf-42a6f1f8a7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.233177 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15ade2-3cf3-4065-a6cf-42a6f1f8a7db admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.243710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 829/1661] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:36.247678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15ade2-3cf3-4065-a6cf-42a6f1f8a7db admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.257960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15ade2-3cf3-4065-a6cf-42a6f1f8a7db admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.262580 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2db86e-3316-4537-ad66-096d609bf9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.263161 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2db86e-3316-4537-ad66-096d609bf9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.263433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2db86e-3316-4537-ad66-096d609bf9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.263698 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2db86e-3316-4537-ad66-096d609bf9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.264317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2db86e-3316-4537-ad66-096d609bf9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.265923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2db86e-3316-4537-ad66-096d609bf9f2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.276520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 830/1662] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:36.280766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2db86e-3316-4537-ad66-096d609bf9f2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.284510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c5a4c-175f-48f9-a573-2227c647a2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.286645 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c5a4c-175f-48f9-a573-2227c647a2fc service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.290176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2db86e-3316-4537-ad66-096d609bf9f2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.295069 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5545de-e4e0-4516-8d12-98464ad4d0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.295748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5545de-e4e0-4516-8d12-98464ad4d0b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.296027 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5545de-e4e0-4516-8d12-98464ad4d0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.296328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5545de-e4e0-4516-8d12-98464ad4d0b5 admin admin] PATH_INFO: `/v3/projects/9e392760b67f4f8685f95d64a1e8821b/users/7792c486ebeb4827a01306aeb4f64299/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.296523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 833/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:36.311641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5545de-e4e0-4516-8d12-98464ad4d0b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e392760b67f4f8685f95d64a1e8821b, user_id=7792c486ebeb4827a01306aeb4f64299, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.314790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5545de-e4e0-4516-8d12-98464ad4d0b5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.327197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 831/1664] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:36 2022] PUT /identity/v3/projects/9e392760b67f4f8685f95d64a1e8821b/users/7792c486ebeb4827a01306aeb4f64299/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:36.341254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd966c7-3e70-4996-a7f9-b44dfa77c6e1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.341558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd966c7-3e70-4996-a7f9-b44dfa77c6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.341829 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd966c7-3e70-4996-a7f9-b44dfa77c6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.371551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fdd966c7-3e70-4996-a7f9-b44dfa77c6e1 None None] MFA Rules not processed for user `7792c486ebeb4827a01306aeb4f64299`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.372781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fdd966c7-3e70-4996-a7f9-b44dfa77c6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.377847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5545de-e4e0-4516-8d12-98464ad4d0b5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.440524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5545de-e4e0-4516-8d12-98464ad4d0b5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.444648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.445495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.445830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.446239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.448209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.467300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 834/1665] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:36.472317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd966c7-3e70-4996-a7f9-b44dfa77c6e1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.482251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:36.516151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.517595 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.521252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:36.534262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 832/1666] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:36.551052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd966c7-3e70-4996-a7f9-b44dfa77c6e1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.555816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf0f84-3f40-4098-a469-f16c0250c181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.556393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf0f84-3f40-4098-a469-f16c0250c181 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.556682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf0f84-3f40-4098-a469-f16c0250c181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.556956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf0f84-3f40-4098-a469-f16c0250c181 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.557794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf0f84-3f40-4098-a469-f16c0250c181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.561119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf0f84-3f40-4098-a469-f16c0250c181 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.601654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 835/1667] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:36.605834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.626225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6ae14-d957-49cf-a98d-4e1b76f17e7e service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.685267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457fb4d-2d36-41f6-9333-537bdfda5e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.685940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457fb4d-2d36-41f6-9333-537bdfda5e9b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.686269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457fb4d-2d36-41f6-9333-537bdfda5e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.686578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457fb4d-2d36-41f6-9333-537bdfda5e9b admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.687267 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457fb4d-2d36-41f6-9333-537bdfda5e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.689324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457fb4d-2d36-41f6-9333-537bdfda5e9b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.713342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 833/1668] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:36.717432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf0f84-3f40-4098-a469-f16c0250c181 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.728481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf0f84-3f40-4098-a469-f16c0250c181 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.738345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d09e1b-1228-43aa-bf40-9a97056ce986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.739059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d09e1b-1228-43aa-bf40-9a97056ce986 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.739720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d09e1b-1228-43aa-bf40-9a97056ce986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.740017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d09e1b-1228-43aa-bf40-9a97056ce986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.740958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d09e1b-1228-43aa-bf40-9a97056ce986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.743675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d09e1b-1228-43aa-bf40-9a97056ce986 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.759141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 836/1669] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:36.760229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457fb4d-2d36-41f6-9333-537bdfda5e9b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.782910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457fb4d-2d36-41f6-9333-537bdfda5e9b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.789441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41c926-9888-49de-9c90-d98dd6f868d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.790235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41c926-9888-49de-9c90-d98dd6f868d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.790630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41c926-9888-49de-9c90-d98dd6f868d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.791151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41c926-9888-49de-9c90-d98dd6f868d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.792037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41c926-9888-49de-9c90-d98dd6f868d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.794052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41c926-9888-49de-9c90-d98dd6f868d8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.813977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 834/1670] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:36.817777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d09e1b-1228-43aa-bf40-9a97056ce986 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.831735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d09e1b-1228-43aa-bf40-9a97056ce986 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.839598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca898-df5e-4c35-9061-b16c3c9b6398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.840286 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca898-df5e-4c35-9061-b16c3c9b6398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.840564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca898-df5e-4c35-9061-b16c3c9b6398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.840865 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca898-df5e-4c35-9061-b16c3c9b6398 admin admin] PATH_INFO: `/v3/projects/c6b481367d1c4a9c8f4b05dce9160d21/users/c2a02b40c76241e28a011e4deab37b14/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.871960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ca898-df5e-4c35-9061-b16c3c9b6398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b481367d1c4a9c8f4b05dce9160d21, user_id=c2a02b40c76241e28a011e4deab37b14, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.874994 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ca898-df5e-4c35-9061-b16c3c9b6398 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.900728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 837/1671] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:36 2022] PUT /identity/v3/projects/c6b481367d1c4a9c8f4b05dce9160d21/users/c2a02b40c76241e28a011e4deab37b14/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:36.903676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc88a-88c0-43cd-aa75-8a3b9cfdca0b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.903966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc88a-88c0-43cd-aa75-8a3b9cfdca0b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.904244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc88a-88c0-43cd-aa75-8a3b9cfdca0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.946446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c74fc88a-88c0-43cd-aa75-8a3b9cfdca0b None None] MFA Rules not processed for user `c2a02b40c76241e28a011e4deab37b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.947581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c74fc88a-88c0-43cd-aa75-8a3b9cfdca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:37.030038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 835/1672] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:37.034914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca898-df5e-4c35-9061-b16c3c9b6398 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.106673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca898-df5e-4c35-9061-b16c3c9b6398 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.111457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39753966-8c3a-4703-8ef5-8a4b3bfc4b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.112067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753966-8c3a-4703-8ef5-8a4b3bfc4b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.112349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753966-8c3a-4703-8ef5-8a4b3bfc4b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.112620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753966-8c3a-4703-8ef5-8a4b3bfc4b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.113272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39753966-8c3a-4703-8ef5-8a4b3bfc4b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.115237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39753966-8c3a-4703-8ef5-8a4b3bfc4b54 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.144952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 838/1673] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:37 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:37.148935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fc88a-88c0-43cd-aa75-8a3b9cfdca0b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.160169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fc88a-88c0-43cd-aa75-8a3b9cfdca0b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.226422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e85787-b07c-46b6-8eea-30a32b021327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.227083 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e85787-b07c-46b6-8eea-30a32b021327 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.227367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e85787-b07c-46b6-8eea-30a32b021327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.227641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e85787-b07c-46b6-8eea-30a32b021327 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.228290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e85787-b07c-46b6-8eea-30a32b021327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.230231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e85787-b07c-46b6-8eea-30a32b021327 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.243128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:37.255658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 836/1674] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:37 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:37.259744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39753966-8c3a-4703-8ef5-8a4b3bfc4b54 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.270102 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39753966-8c3a-4703-8ef5-8a4b3bfc4b54 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.274738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5585398-8b5a-4754-bf97-5562cd26ef81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.275345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5585398-8b5a-4754-bf97-5562cd26ef81 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.275626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5585398-8b5a-4754-bf97-5562cd26ef81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.275918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5585398-8b5a-4754-bf97-5562cd26ef81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.276557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5585398-8b5a-4754-bf97-5562cd26ef81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.278185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5585398-8b5a-4754-bf97-5562cd26ef81 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.289566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 839/1675] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:37.293695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e85787-b07c-46b6-8eea-30a32b021327 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.303836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e85787-b07c-46b6-8eea-30a32b021327 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.309194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3329ca2-43b9-41c2-891b-c09602d612bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.309834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329ca2-43b9-41c2-891b-c09602d612bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.310284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329ca2-43b9-41c2-891b-c09602d612bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.310577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329ca2-43b9-41c2-891b-c09602d612bc admin admin] PATH_INFO: `/v3/projects/98cba1581ae34d5fb0377798bdacdf8a/users/ac65638aa53d4304a1d41179a02b2204/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.325684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329ca2-43b9-41c2-891b-c09602d612bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cba1581ae34d5fb0377798bdacdf8a, user_id=ac65638aa53d4304a1d41179a02b2204, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.331243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329ca2-43b9-41c2-891b-c09602d612bc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.343156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 837/1676] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:37 2022] PUT /identity/v3/projects/98cba1581ae34d5fb0377798bdacdf8a/users/ac65638aa53d4304a1d41179a02b2204/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:37.347784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5585398-8b5a-4754-bf97-5562cd26ef81 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.358932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5585398-8b5a-4754-bf97-5562cd26ef81 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.395747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:37.426654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68819dd-aaa8-43a0-8019-a185cbf9ba29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.426989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68819dd-aaa8-43a0-8019-a185cbf9ba29 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.427265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68819dd-aaa8-43a0-8019-a185cbf9ba29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.427532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68819dd-aaa8-43a0-8019-a185cbf9ba29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.428168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68819dd-aaa8-43a0-8019-a185cbf9ba29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.429761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68819dd-aaa8-43a0-8019-a185cbf9ba29 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.443533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 840/1677] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:37.447816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3329ca2-43b9-41c2-891b-c09602d612bc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.461047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3329ca2-43b9-41c2-891b-c09602d612bc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.468155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d049778-002c-44eb-b385-6fd06cfd5a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.470298 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049778-002c-44eb-b385-6fd06cfd5a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.470619 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049778-002c-44eb-b385-6fd06cfd5a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.470893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049778-002c-44eb-b385-6fd06cfd5a57 admin admin] PATH_INFO: `/v3/projects/98cba1581ae34d5fb0377798bdacdf8a/users/ac65638aa53d4304a1d41179a02b2204/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.472878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d049778-002c-44eb-b385-6fd06cfd5a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cba1581ae34d5fb0377798bdacdf8a, user_id=ac65638aa53d4304a1d41179a02b2204, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.475594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d049778-002c-44eb-b385-6fd06cfd5a57 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.495660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 838/1678] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:37 2022] PUT /identity/v3/projects/98cba1581ae34d5fb0377798bdacdf8a/users/ac65638aa53d4304a1d41179a02b2204/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:37.498527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a796e83-653d-4575-b876-d843b0cf2549 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.498924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a796e83-653d-4575-b876-d843b0cf2549 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.499390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a796e83-653d-4575-b876-d843b0cf2549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.529148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8a796e83-653d-4575-b876-d843b0cf2549 None None] MFA Rules not processed for user `ac65638aa53d4304a1d41179a02b2204`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:37.530402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a796e83-653d-4575-b876-d843b0cf2549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:37.604649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 841/1679] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:37.616617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d049778-002c-44eb-b385-6fd06cfd5a57 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.691534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d049778-002c-44eb-b385-6fd06cfd5a57 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.696626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8478944-44b8-448e-ada7-8936a3ec0eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.697247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8478944-44b8-448e-ada7-8936a3ec0eae service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.697521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8478944-44b8-448e-ada7-8936a3ec0eae service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.697794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8478944-44b8-448e-ada7-8936a3ec0eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.759590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8478944-44b8-448e-ada7-8936a3ec0eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.761365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8478944-44b8-448e-ada7-8936a3ec0eae service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.770878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 839/1680] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:39.003486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4552381-38af-4eab-a449-ba0cbb422f7e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.004558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4552381-38af-4eab-a449-ba0cbb422f7e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.005199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4552381-38af-4eab-a449-ba0cbb422f7e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.009242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 842/1681] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:28:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:39.020505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac156-55a6-44e3-acd1-b630184505c5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.021347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac156-55a6-44e3-acd1-b630184505c5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.021815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac156-55a6-44e3-acd1-b630184505c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.054281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-962ac156-55a6-44e3-acd1-b630184505c5 None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:39.055843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-962ac156-55a6-44e3-acd1-b630184505c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.150994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 840/1682] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:39.253823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4552381-38af-4eab-a449-ba0cbb422f7e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.256238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d4552381-38af-4eab-a449-ba0cbb422f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.310698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4552381-38af-4eab-a449-ba0cbb422f7e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.322442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ac156-55a6-44e3-acd1-b630184505c5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.324036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-962ac156-55a6-44e3-acd1-b630184505c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.338549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:39.340260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.347109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.348004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.348312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.385061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.386987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.414759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 843/1683] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:39.483316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.484854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.497905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3484a1-2bb6-48d0-a557-1bf58160aacc service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.504801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97f5f5-898e-4791-ac4d-daf378d63d4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.505839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97f5f5-898e-4791-ac4d-daf378d63d4f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.506240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97f5f5-898e-4791-ac4d-daf378d63d4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.507000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97f5f5-898e-4791-ac4d-daf378d63d4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.524704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ac156-55a6-44e3-acd1-b630184505c5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.530121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.531059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.531344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.531727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] PATH_INFO: `/v3/users/e9a83e6dc5ed443f908da3a1ca8ecc87` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.544402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a83e6dc5ed443f908da3a1ca8ecc87)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.546505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.591969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97f5f5-898e-4791-ac4d-daf378d63d4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.593524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97f5f5-898e-4791-ac4d-daf378d63d4f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.621578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 844/1684] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:39.649692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a83e6dc5ed443f908da3a1ca8ecc87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.654648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a83e6dc5ed443f908da3a1ca8ecc87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.676710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a83e6dc5ed443f908da3a1ca8ecc87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.687655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a83e6dc5ed443f908da3a1ca8ecc87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.688937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a83e6dc5ed443f908da3a1ca8ecc87'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.706592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 841/1685] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:39 2022] DELETE /identity/v3/users/e9a83e6dc5ed443f908da3a1ca8ecc87 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:39.710875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50079-ece8-47a8-801d-70b78a0bf0cd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.711399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50079-ece8-47a8-801d-70b78a0bf0cd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.711966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50079-ece8-47a8-801d-70b78a0bf0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.749334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e7c50079-ece8-47a8-801d-70b78a0bf0cd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:39.749334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7c50079-ece8-47a8-801d-70b78a0bf0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.806359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:39.836473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 845/1686] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:39.847236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.848635 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.919111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd52d16-7ab1-4cb4-934f-0309dd82185c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.923185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9156e6-aa60-42a7-a9d0-2987df85dda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.923949 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9156e6-aa60-42a7-a9d0-2987df85dda9 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.924217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9156e6-aa60-42a7-a9d0-2987df85dda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.924529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9156e6-aa60-42a7-a9d0-2987df85dda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.934411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9156e6-aa60-42a7-a9d0-2987df85dda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.935872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9156e6-aa60-42a7-a9d0-2987df85dda9 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.948904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 842/1687] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:39.989378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50079-ece8-47a8-801d-70b78a0bf0cd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.990904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7c50079-ece8-47a8-801d-70b78a0bf0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.003724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50079-ece8-47a8-801d-70b78a0bf0cd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.008406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.010115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.010412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.010741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] PATH_INFO: `/v3/users/6579c55c435a46c3bea93f73e76c6a15` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.011896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6579c55c435a46c3bea93f73e76c6a15)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.013942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.074299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6579c55c435a46c3bea93f73e76c6a15'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.075188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6579c55c435a46c3bea93f73e76c6a15'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.092815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6579c55c435a46c3bea93f73e76c6a15'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.103020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6579c55c435a46c3bea93f73e76c6a15'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.103946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6579c55c435a46c3bea93f73e76c6a15'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.119728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 846/1688] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:39 2022] DELETE /identity/v3/users/6579c55c435a46c3bea93f73e76c6a15 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:40.125086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a19fa-12d2-4713-a52e-83a8ebbff783 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.125558 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a19fa-12d2-4713-a52e-83a8ebbff783 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.125765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a19fa-12d2-4713-a52e-83a8ebbff783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.148023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-673a19fa-12d2-4713-a52e-83a8ebbff783 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:40.149625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-673a19fa-12d2-4713-a52e-83a8ebbff783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.174982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:40.226909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 843/1689] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:40.240981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.243110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.276295 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a19fa-12d2-4713-a52e-83a8ebbff783 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.277902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-673a19fa-12d2-4713-a52e-83a8ebbff783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.300914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a19fa-12d2-4713-a52e-83a8ebbff783 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.311026 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.312355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.312925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.313646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] PATH_INFO: `/v3/projects/9698231c75fb47f693f7dcd0ced9e97d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.315542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9698231c75fb47f693f7dcd0ced9e97d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.318701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.320265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c567d1-c0e3-46e6-87e5-1ece04a9bd55 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.324323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.324934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.325210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.325479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.336946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.338446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.354728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 847/1690] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:40.355999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.357503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.406034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9698231c75fb47f693f7dcd0ced9e97d', 'request_id': 'req-1edc55c6-729b-40ba-b376-b946c0a557cd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.420404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9698231c75fb47f693f7dcd0ced9e97d', 'request_id': 'req-1edc55c6-729b-40ba-b376-b946c0a557cd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.428302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9698231c75fb47f693f7dcd0ced9e97d', 'request_id': 'req-1edc55c6-729b-40ba-b376-b946c0a557cd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.437588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4094f-530b-4301-b552-04e8c81df9fd service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.442787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] The token cache is being invalidate because project 9698231c75fb47f693f7dcd0ced9e97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:40.443560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.443886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1edc55c6-729b-40ba-b376-b946c0a557cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.445715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 844/1691] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:40 2022] DELETE /identity/v3/projects/9698231c75fb47f693f7dcd0ced9e97d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:40.450385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:40.451323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845956ab-6601-4c5d-930f-47c7a180bc54 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.451754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845956ab-6601-4c5d-930f-47c7a180bc54 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.452820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845956ab-6601-4c5d-930f-47c7a180bc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.479797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-845956ab-6601-4c5d-930f-47c7a180bc54 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:40.480471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-845956ab-6601-4c5d-930f-47c7a180bc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.502689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.503308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.503588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.503859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.505784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.569547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 845/1692] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:40.576693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf57c-b5f5-4672-9b6d-3d1a943fd4ff None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.577224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf57c-b5f5-4672-9b6d-3d1a943fd4ff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.577655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf57c-b5f5-4672-9b6d-3d1a943fd4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.599660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.601227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.603442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-455bf57c-b5f5-4672-9b6d-3d1a943fd4ff None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:40.604840 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-455bf57c-b5f5-4672-9b6d-3d1a943fd4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.619812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 846/1693] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:40.620253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 848/1694] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:40.628160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bf57c-b5f5-4672-9b6d-3d1a943fd4ff None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.641966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bf57c-b5f5-4672-9b6d-3d1a943fd4ff None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.647554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.648227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.648718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.649400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.650166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:40.650686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.652850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.669400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 847/1695] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:40.670711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.683125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b98f-c1b0-4ddd-a388-ebd46d6df36b service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.688261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.688896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.689351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.689494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.690221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.694274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.723975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 849/1696] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:40 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:40.727737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.738925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845697cc-29b5-4bfb-b6c4-e3acd271edbb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.775892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:40.822693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.824988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.841881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ba2db-e8bc-4f35-86e9-d287ddefcef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.842670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba2db-e8bc-4f35-86e9-d287ddefcef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.842946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba2db-e8bc-4f35-86e9-d287ddefcef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.843215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba2db-e8bc-4f35-86e9-d287ddefcef7 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.844158 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ba2db-e8bc-4f35-86e9-d287ddefcef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.846396 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ba2db-e8bc-4f35-86e9-d287ddefcef7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.877286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 848/1697] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:40 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:40.883419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ba2db-e8bc-4f35-86e9-d287ddefcef7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.897953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ba2db-e8bc-4f35-86e9-d287ddefcef7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.907442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b880a3-ee45-46d5-8378-21a7fbee13de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.908333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b880a3-ee45-46d5-8378-21a7fbee13de admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.908868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b880a3-ee45-46d5-8378-21a7fbee13de admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.909557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b880a3-ee45-46d5-8378-21a7fbee13de admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.910901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b880a3-ee45-46d5-8378-21a7fbee13de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.912922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b880a3-ee45-46d5-8378-21a7fbee13de admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.928517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 849/1698] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:40.932618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b880a3-ee45-46d5-8378-21a7fbee13de admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.947490 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b880a3-ee45-46d5-8378-21a7fbee13de admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.953714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512303fb-5026-48f3-af07-6f003c570254 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.957986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a221f-c81b-4365-8c83-6f774f90d56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.958822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a221f-c81b-4365-8c83-6f774f90d56b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.959264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a221f-c81b-4365-8c83-6f774f90d56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.959526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.960335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.960677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.960987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] PATH_INFO: `/v3/projects/7aee503d682b4c19b47c632d890908e6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.961169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a221f-c81b-4365-8c83-6f774f90d56b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.961999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a221f-c81b-4365-8c83-6f774f90d56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.962325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aee503d682b4c19b47c632d890908e6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.964485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.965064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a221f-c81b-4365-8c83-6f774f90d56b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.984034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 850/1699] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:40.990145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a221f-c81b-4365-8c83-6f774f90d56b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.011079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a221f-c81b-4365-8c83-6f774f90d56b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.026493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74febef-ae3d-4c79-ab40-f06b516a4cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.029411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74febef-ae3d-4c79-ab40-f06b516a4cee admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.029637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74febef-ae3d-4c79-ab40-f06b516a4cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.030020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74febef-ae3d-4c79-ab40-f06b516a4cee admin admin] PATH_INFO: `/v3/projects/b10aabcbed534f298d66ac23b519b1f9/users/fe23b30102114521b7bc797c7a57fd28/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.045422 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:41.058079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74febef-ae3d-4c79-ab40-f06b516a4cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10aabcbed534f298d66ac23b519b1f9, user_id=fe23b30102114521b7bc797c7a57fd28, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.063092 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aee503d682b4c19b47c632d890908e6', 'request_id': 'req-176fc801-c1ea-4b8b-8a1a-b49a762bda25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.063497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74febef-ae3d-4c79-ab40-f06b516a4cee admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.085412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 851/1700] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:40 2022] PUT /identity/v3/projects/b10aabcbed534f298d66ac23b519b1f9/users/fe23b30102114521b7bc797c7a57fd28/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:41.098318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cee535-a1ec-49d6-a953-4a910b6ffc37 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.098851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cee535-a1ec-49d6-a953-4a910b6ffc37 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.099263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cee535-a1ec-49d6-a953-4a910b6ffc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.101593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aee503d682b4c19b47c632d890908e6', 'request_id': 'req-176fc801-c1ea-4b8b-8a1a-b49a762bda25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.116811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aee503d682b4c19b47c632d890908e6', 'request_id': 'req-176fc801-c1ea-4b8b-8a1a-b49a762bda25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.133653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] The token cache is being invalidate because project 7aee503d682b4c19b47c632d890908e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:41.134003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.135140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.136860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 850/1701] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:40 2022] DELETE /identity/v3/projects/7aee503d682b4c19b47c632d890908e6 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:41.137766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.140037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.144142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-05cee535-a1ec-49d6-a953-4a910b6ffc37 None None] MFA Rules not processed for user `fe23b30102114521b7bc797c7a57fd28`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:41.145954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-05cee535-a1ec-49d6-a953-4a910b6ffc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.230872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc801-c1ea-4b8b-8a1a-b49a762bda25 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.241985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 852/1702] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:41.242752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cee535-a1ec-49d6-a953-4a910b6ffc37 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.244549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-05cee535-a1ec-49d6-a953-4a910b6ffc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.305730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.306401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.306680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.306951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] PATH_INFO: `/v3/users/bf33e3330d5c4de1952c2878164090ed` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.308089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf33e3330d5c4de1952c2878164090ed)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.310018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.316044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cee535-a1ec-49d6-a953-4a910b6ffc37 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.320608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.321235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.321508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.321781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] PATH_INFO: `/v3/users/88c7d185b11349a49447002ffb97a12f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.322919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c7d185b11349a49447002ffb97a12f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.324817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.329957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:41.358991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf33e3330d5c4de1952c2878164090ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.359716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf33e3330d5c4de1952c2878164090ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.373986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c7d185b11349a49447002ffb97a12f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.374204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf33e3330d5c4de1952c2878164090ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.374669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c7d185b11349a49447002ffb97a12f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.381515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf33e3330d5c4de1952c2878164090ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.382361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf33e3330d5c4de1952c2878164090ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.386194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:41.386798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c7d185b11349a49447002ffb97a12f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.393916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c7d185b11349a49447002ffb97a12f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.394642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2fabd77e-140b-466c-bebd-2cd86872231f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c7d185b11349a49447002ffb97a12f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.397244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 851/1703] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:41 2022] DELETE /identity/v3/users/bf33e3330d5c4de1952c2878164090ed => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:41.399044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.401484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.408997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 853/1704] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:41 2022] DELETE /identity/v3/users/88c7d185b11349a49447002ffb97a12f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:41.412798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16982545-994e-48f4-a9f7-ad40690bbadb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.412798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16982545-994e-48f4-a9f7-ad40690bbadb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.412798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16982545-994e-48f4-a9f7-ad40690bbadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.434411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-16982545-994e-48f4-a9f7-ad40690bbadb None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:41.435950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-16982545-994e-48f4-a9f7-ad40690bbadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.500268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d26124-b8f9-4b03-92d1-b0a35dab13df admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.510214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 854/1705] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:41.522560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16982545-994e-48f4-a9f7-ad40690bbadb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.524034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-16982545-994e-48f4-a9f7-ad40690bbadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.563663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517c200-0820-418e-aa23-3ada99788f16 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fd09151e284f639c93699b0d26db05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.564280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517c200-0820-418e-aa23-3ada99788f16 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.564574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517c200-0820-418e-aa23-3ada99788f16 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.564990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517c200-0820-418e-aa23-3ada99788f16 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.569001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c517c200-0820-418e-aa23-3ada99788f16 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.610955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16982545-994e-48f4-a9f7-ad40690bbadb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.615121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.615916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.616238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.616563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.635565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.637375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.662143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 855/1706] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:41.681619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517c200-0820-418e-aa23-3ada99788f16 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.683310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517c200-0820-418e-aa23-3ada99788f16 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.695642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 852/1707] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:41.888125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.889705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.933382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e67eb-88f1-4da6-b653-7dd3687726c4 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.946423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.947072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.947355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.947629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] PATH_INFO: `/v3/projects/940655e9818c42178c82b44ee11b7587` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.950366 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940655e9818c42178c82b44ee11b7587)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.952473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.008713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:42.063542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940655e9818c42178c82b44ee11b7587', 'request_id': 'req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.081176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940655e9818c42178c82b44ee11b7587', 'request_id': 'req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.090858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940655e9818c42178c82b44ee11b7587', 'request_id': 'req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.115984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] The token cache is being invalidate because project 940655e9818c42178c82b44ee11b7587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:42.116380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.117326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4c78403c-1cdf-4ee5-b8c3-649db0727bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.119363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 856/1708] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:41 2022] DELETE /identity/v3/projects/940655e9818c42178c82b44ee11b7587 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:42.124378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f971871-a3d3-4904-acec-ef84e0e2156c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.124882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f971871-a3d3-4904-acec-ef84e0e2156c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.125412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f971871-a3d3-4904-acec-ef84e0e2156c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.152321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1f971871-a3d3-4904-acec-ef84e0e2156c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:42.153642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f971871-a3d3-4904-acec-ef84e0e2156c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.205708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:42.241487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 853/1709] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:42.254618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00bb6a-7f76-450f-9292-35b6e332758d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.255163 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00bb6a-7f76-450f-9292-35b6e332758d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.255554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00bb6a-7f76-450f-9292-35b6e332758d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.270768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f971871-a3d3-4904-acec-ef84e0e2156c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.272422 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f971871-a3d3-4904-acec-ef84e0e2156c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.284638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3f00bb6a-7f76-450f-9292-35b6e332758d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:42.285905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f00bb6a-7f76-450f-9292-35b6e332758d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.287169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f971871-a3d3-4904-acec-ef84e0e2156c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.294824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.295517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.295794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.296062 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] PATH_INFO: `/v3/projects/19831341772a4015927c029f947030aa` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.297491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] RBAC: Authorizing `identity:delete_project(project_id=19831341772a4015927c029f947030aa)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.299754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.300117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 857/1710] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:42.301032 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00bb6a-7f76-450f-9292-35b6e332758d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.313620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00bb6a-7f76-450f-9292-35b6e332758d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.340157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.340858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.348422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.348911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.349738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:42.350743 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.352715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.372817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 858/1711] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:42.377646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.389043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becebcd-b79f-4e89-8dac-aba8a439c60a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.416904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19831341772a4015927c029f947030aa', 'request_id': 'req-50ecd387-b0b1-4c53-88d6-3d6987c4658f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.435642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19831341772a4015927c029f947030aa', 'request_id': 'req-50ecd387-b0b1-4c53-88d6-3d6987c4658f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.453928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19831341772a4015927c029f947030aa', 'request_id': 'req-50ecd387-b0b1-4c53-88d6-3d6987c4658f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.478850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] The token cache is being invalidate because project 19831341772a4015927c029f947030aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:42.479423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.480145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.481756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 854/1712] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:42 2022] DELETE /identity/v3/projects/19831341772a4015927c029f947030aa => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:42.486463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.488531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.535005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.535650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.535928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.536248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.537745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.540478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.586468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecd387-b0b1-4c53-88d6-3d6987c4658f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.588491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 859/1713] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:42.592862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.595886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.650484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.651459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.652083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.652559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] PATH_INFO: `/v3/users/db5bf03605f74b9580711a746107db65` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.655535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5bf03605f74b9580711a746107db65)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.657801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.681341 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69dc0b0-02ce-4cd6-9f68-c664c8c8328d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.686508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:42.692681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.693605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.694075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.694560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.695504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.697503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.721572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5bf03605f74b9580711a746107db65'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.722573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5bf03605f74b9580711a746107db65'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.723727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 860/1714] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:42.727744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.729355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.739821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:42.740071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5bf03605f74b9580711a746107db65'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.744987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541338a3-bf59-4854-8c22-85c50a0d5490 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.747201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5bf03605f74b9580711a746107db65'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.748103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5bf03605f74b9580711a746107db65'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.754990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930da15d-2ff5-4e0b-a5ee-23bf9c121ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.755967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da15d-2ff5-4e0b-a5ee-23bf9c121ace admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.756285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da15d-2ff5-4e0b-a5ee-23bf9c121ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.756613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da15d-2ff5-4e0b-a5ee-23bf9c121ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.757255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930da15d-2ff5-4e0b-a5ee-23bf9c121ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.759117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930da15d-2ff5-4e0b-a5ee-23bf9c121ace admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.763455 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 855/1715] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:42 2022] DELETE /identity/v3/users/db5bf03605f74b9580711a746107db65 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:42.774995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 861/1716] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:42.776492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.843617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae535ef1-29ff-4299-bb4d-de893657bc49 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.850047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451ebc1-faa3-4d0c-ab90-2be669884b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.850715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451ebc1-faa3-4d0c-ab90-2be669884b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.850998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451ebc1-faa3-4d0c-ab90-2be669884b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.851488 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451ebc1-faa3-4d0c-ab90-2be669884b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.851944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451ebc1-faa3-4d0c-ab90-2be669884b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.853700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451ebc1-faa3-4d0c-ab90-2be669884b5a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.864789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 856/1717] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:42.869487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930da15d-2ff5-4e0b-a5ee-23bf9c121ace admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.879906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930da15d-2ff5-4e0b-a5ee-23bf9c121ace admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.884447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.885103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.885379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.885648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] PATH_INFO: `/v3/projects/1d3fbb20c86e4f9cb73273144f1734d9/users/1d750b1b27d840b38fd541653a61640b/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.906054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3fbb20c86e4f9cb73273144f1734d9, user_id=1d750b1b27d840b38fd541653a61640b, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.908896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.941881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 862/1718] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:42 2022] PUT /identity/v3/projects/1d3fbb20c86e4f9cb73273144f1734d9/users/1d750b1b27d840b38fd541653a61640b/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:42.942012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602102e1-989b-4b1f-a04c-dc026ebe32fe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.942687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602102e1-989b-4b1f-a04c-dc026ebe32fe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.942687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602102e1-989b-4b1f-a04c-dc026ebe32fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.966405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-602102e1-989b-4b1f-a04c-dc026ebe32fe None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:42.968458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-602102e1-989b-4b1f-a04c-dc026ebe32fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.073681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 857/1719] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:43.075218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.078039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.093155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab89c52-af52-4b87-8943-fe9042f9777d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.099444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.100305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.101380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.101803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] PATH_INFO: `/v3/projects/d2abcef2998f42e5a049e3c98aec6dcb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.103306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2abcef2998f42e5a049e3c98aec6dcb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.105419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602102e1-989b-4b1f-a04c-dc026ebe32fe None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.107425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-602102e1-989b-4b1f-a04c-dc026ebe32fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.115968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.162298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:43.203676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2abcef2998f42e5a049e3c98aec6dcb', 'request_id': 'req-4a3f4a09-978c-40b7-9c22-1e6df79ee977'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.217747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602102e1-989b-4b1f-a04c-dc026ebe32fe None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.218215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2abcef2998f42e5a049e3c98aec6dcb', 'request_id': 'req-4a3f4a09-978c-40b7-9c22-1e6df79ee977'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.228615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2abcef2998f42e5a049e3c98aec6dcb', 'request_id': 'req-4a3f4a09-978c-40b7-9c22-1e6df79ee977'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.247839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] The token cache is being invalidate because project d2abcef2998f42e5a049e3c98aec6dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:43.248933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.249748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4a3f4a09-978c-40b7-9c22-1e6df79ee977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.256133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 863/1720] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:43 2022] DELETE /identity/v3/projects/d2abcef2998f42e5a049e3c98aec6dcb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:43.258849 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c480-f34a-40fe-a571-36544fcbd5f6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.259280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c480-f34a-40fe-a571-36544fcbd5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.259598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c480-f34a-40fe-a571-36544fcbd5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.299494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3df5c480-f34a-40fe-a571-36544fcbd5f6 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:43.301721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3df5c480-f34a-40fe-a571-36544fcbd5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.312954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.314442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.314817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.315195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.321263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.397241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 864/1721] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:43.400864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fdb4d-5130-49e6-90e1-47a7b1c2342c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.401145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fdb4d-5130-49e6-90e1-47a7b1c2342c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.401419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fdb4d-5130-49e6-90e1-47a7b1c2342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.426501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-aa4fdb4d-5130-49e6-90e1-47a7b1c2342c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:43.427845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aa4fdb4d-5130-49e6-90e1-47a7b1c2342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.439882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 865/1722] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:43.441838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fdb4d-5130-49e6-90e1-47a7b1c2342c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.461747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fdb4d-5130-49e6-90e1-47a7b1c2342c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.465025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.467032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.467253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.467901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.468140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.468414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.470582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:43.470895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.472760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.484061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 858/1723] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:43.490442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 866/1724] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:43.495153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.541799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dd830-cb13-4e55-806d-e59730ccd39f service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.546958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686de47c-b004-4c46-9170-ca3e1902a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.547940 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686de47c-b004-4c46-9170-ca3e1902a434 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.548274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686de47c-b004-4c46-9170-ca3e1902a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.548563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686de47c-b004-4c46-9170-ca3e1902a434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.549380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686de47c-b004-4c46-9170-ca3e1902a434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.554867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686de47c-b004-4c46-9170-ca3e1902a434 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.632760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.648123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 859/1725] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:43 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:43.654750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8f5a0-f951-4349-afae-e11f2233e077 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.708804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:43.770485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ba30d-ca92-4457-a12c-ced96e2b1ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.771277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ba30d-ca92-4457-a12c-ced96e2b1ebd admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.771569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ba30d-ca92-4457-a12c-ced96e2b1ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.774502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ba30d-ca92-4457-a12c-ced96e2b1ebd admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.775203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ba30d-ca92-4457-a12c-ced96e2b1ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.777137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ba30d-ca92-4457-a12c-ced96e2b1ebd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.844271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 867/1726] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:43 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:43.845653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686de47c-b004-4c46-9170-ca3e1902a434 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.870191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686de47c-b004-4c46-9170-ca3e1902a434 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.888212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be874d6-095d-40c1-85f0-6fe26eaa3ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.888839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be874d6-095d-40c1-85f0-6fe26eaa3ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.889113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be874d6-095d-40c1-85f0-6fe26eaa3ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.889379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be874d6-095d-40c1-85f0-6fe26eaa3ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.890024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be874d6-095d-40c1-85f0-6fe26eaa3ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.891789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be874d6-095d-40c1-85f0-6fe26eaa3ebf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.918396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 860/1727] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:43.921662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ba30d-ca92-4457-a12c-ced96e2b1ebd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.933341 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ba30d-ca92-4457-a12c-ced96e2b1ebd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.938154 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e6fcc-71a8-4518-9215-2b7632225d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.939039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e6fcc-71a8-4518-9215-2b7632225d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.939339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e6fcc-71a8-4518-9215-2b7632225d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.939948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e6fcc-71a8-4518-9215-2b7632225d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.940463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e6fcc-71a8-4518-9215-2b7632225d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.943530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e6fcc-71a8-4518-9215-2b7632225d6a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.961068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be874d6-095d-40c1-85f0-6fe26eaa3ebf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.963236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 868/1728] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:43.974320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be874d6-095d-40c1-85f0-6fe26eaa3ebf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.980494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ac676-fa1b-4fab-b059-a1678c8053ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.981679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac676-fa1b-4fab-b059-a1678c8053ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.982144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac676-fa1b-4fab-b059-a1678c8053ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.982565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac676-fa1b-4fab-b059-a1678c8053ed admin admin] PATH_INFO: `/v3/projects/8ff207dc6bef49c68ab81eaf90ae71a9/users/5edb902b5d464592ae405adf19c0f358/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.025861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ac676-fa1b-4fab-b059-a1678c8053ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff207dc6bef49c68ab81eaf90ae71a9, user_id=5edb902b5d464592ae405adf19c0f358, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.028967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ac676-fa1b-4fab-b059-a1678c8053ed admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.070260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 861/1729] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:43 2022] PUT /identity/v3/projects/8ff207dc6bef49c68ab81eaf90ae71a9/users/5edb902b5d464592ae405adf19c0f358/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:44.083273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a24e-23b6-4bb1-aba5-df981291d355 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.083739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a24e-23b6-4bb1-aba5-df981291d355 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.084841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a24e-23b6-4bb1-aba5-df981291d355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.130577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4e20a24e-23b6-4bb1-aba5-df981291d355 None None] MFA Rules not processed for user `5edb902b5d464592ae405adf19c0f358`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:44.132941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4e20a24e-23b6-4bb1-aba5-df981291d355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.246163 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 869/1730] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:44.246634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ac676-fa1b-4fab-b059-a1678c8053ed admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.319081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:44.353955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ac676-fa1b-4fab-b059-a1678c8053ed admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.374640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d9eb2-b088-4e17-9386-dded0a6a8967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.377067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d9eb2-b088-4e17-9386-dded0a6a8967 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.377480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d9eb2-b088-4e17-9386-dded0a6a8967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.377915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d9eb2-b088-4e17-9386-dded0a6a8967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.379301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d9eb2-b088-4e17-9386-dded0a6a8967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.381739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d9eb2-b088-4e17-9386-dded0a6a8967 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.435557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 862/1731] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:44 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:44.439744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20a24e-23b6-4bb1-aba5-df981291d355 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.453552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20a24e-23b6-4bb1-aba5-df981291d355 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.557010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140955df-ae92-47f4-b40d-dde9b5a7fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.557910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140955df-ae92-47f4-b40d-dde9b5a7fd18 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.558916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140955df-ae92-47f4-b40d-dde9b5a7fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.559305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140955df-ae92-47f4-b40d-dde9b5a7fd18 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.559970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140955df-ae92-47f4-b40d-dde9b5a7fd18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.562776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140955df-ae92-47f4-b40d-dde9b5a7fd18 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.597332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 870/1732] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:44 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:44.599211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d9eb2-b088-4e17-9386-dded0a6a8967 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.631154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d9eb2-b088-4e17-9386-dded0a6a8967 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.645348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3247-3022-4589-b505-7d0d5f9f1f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.646884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3247-3022-4589-b505-7d0d5f9f1f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.647466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3247-3022-4589-b505-7d0d5f9f1f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.648088 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3247-3022-4589-b505-7d0d5f9f1f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.649505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c3247-3022-4589-b505-7d0d5f9f1f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.653203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c3247-3022-4589-b505-7d0d5f9f1f5b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.676976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140955df-ae92-47f4-b40d-dde9b5a7fd18 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.690304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 863/1733] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:44.691494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140955df-ae92-47f4-b40d-dde9b5a7fd18 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.697351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46d6e7-0b17-4a5f-8014-506156b2527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.698049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46d6e7-0b17-4a5f-8014-506156b2527b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.698496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46d6e7-0b17-4a5f-8014-506156b2527b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.698835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46d6e7-0b17-4a5f-8014-506156b2527b admin admin] PATH_INFO: `/v3/projects/947b45c0e5024703b4fe4b8579be4c21/users/badb315377134771a62a9b02ad18ce37/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.720631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46d6e7-0b17-4a5f-8014-506156b2527b admin admin] RBAC: Authorizing `identity:create_grant(project_id=947b45c0e5024703b4fe4b8579be4c21, user_id=badb315377134771a62a9b02ad18ce37, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.723597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46d6e7-0b17-4a5f-8014-506156b2527b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.758627 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 871/1734] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:44 2022] PUT /identity/v3/projects/947b45c0e5024703b4fe4b8579be4c21/users/badb315377134771a62a9b02ad18ce37/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:44.763684 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3247-3022-4589-b505-7d0d5f9f1f5b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.776743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3247-3022-4589-b505-7d0d5f9f1f5b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.801052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503ce19-b056-4422-a9fe-6f6eaabe5196 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.801522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503ce19-b056-4422-a9fe-6f6eaabe5196 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.801885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503ce19-b056-4422-a9fe-6f6eaabe5196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.840618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:44.871514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9db6-a7af-458e-8cc6-af5083da4c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.872146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9db6-a7af-458e-8cc6-af5083da4c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.872793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9db6-a7af-458e-8cc6-af5083da4c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.873079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9db6-a7af-458e-8cc6-af5083da4c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.873737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc9db6-a7af-458e-8cc6-af5083da4c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.875892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc9db6-a7af-458e-8cc6-af5083da4c59 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.890327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d503ce19-b056-4422-a9fe-6f6eaabe5196 None None] MFA Rules not processed for user `1d750b1b27d840b38fd541653a61640b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:44.891408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 864/1735] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:44.892559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d503ce19-b056-4422-a9fe-6f6eaabe5196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.896392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9db6-a7af-458e-8cc6-af5083da4c59 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.913072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9db6-a7af-458e-8cc6-af5083da4c59 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.926611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a0d2e-bac1-451b-b235-acf75721b8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.927374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0d2e-bac1-451b-b235-acf75721b8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.927710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0d2e-bac1-451b-b235-acf75721b8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.928168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0d2e-bac1-451b-b235-acf75721b8e2 admin admin] PATH_INFO: `/v3/projects/947b45c0e5024703b4fe4b8579be4c21/users/badb315377134771a62a9b02ad18ce37/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.933833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a0d2e-bac1-451b-b235-acf75721b8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=947b45c0e5024703b4fe4b8579be4c21, user_id=badb315377134771a62a9b02ad18ce37, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.936786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a0d2e-bac1-451b-b235-acf75721b8e2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.974032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 865/1736] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:44 2022] PUT /identity/v3/projects/947b45c0e5024703b4fe4b8579be4c21/users/badb315377134771a62a9b02ad18ce37/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:44.978795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5be68-b0c1-4848-b4fe-9e7a6809ef3c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.979086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5be68-b0c1-4848-b4fe-9e7a6809ef3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.979365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5be68-b0c1-4848-b4fe-9e7a6809ef3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.006683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 872/1737] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:45.015168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503ce19-b056-4422-a9fe-6f6eaabe5196 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.016850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d503ce19-b056-4422-a9fe-6f6eaabe5196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.019791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-18f5be68-b0c1-4848-b4fe-9e7a6809ef3c None None] MFA Rules not processed for user `badb315377134771a62a9b02ad18ce37`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:45.021111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-18f5be68-b0c1-4848-b4fe-9e7a6809ef3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.089051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503ce19-b056-4422-a9fe-6f6eaabe5196 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.102082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 866/1738] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:45.113988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5be68-b0c1-4848-b4fe-9e7a6809ef3c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.116064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-18f5be68-b0c1-4848-b4fe-9e7a6809ef3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.148936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:45.149804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79094a06-e012-4539-bb8a-3394a7f31d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.150320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094a06-e012-4539-bb8a-3394a7f31d18 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.150691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094a06-e012-4539-bb8a-3394a7f31d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.151005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094a06-e012-4539-bb8a-3394a7f31d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.204754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5be68-b0c1-4848-b4fe-9e7a6809ef3c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.208916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.209440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.209717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.210014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.218478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79094a06-e012-4539-bb8a-3394a7f31d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.220446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79094a06-e012-4539-bb8a-3394a7f31d18 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.221594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.223112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.233329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 873/1739] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:45.234495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 867/1740] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:45.417726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79094a06-e012-4539-bb8a-3394a7f31d18 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.433641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79094a06-e012-4539-bb8a-3394a7f31d18 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.454703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3a56a-2a5e-476a-90ff-41a5e81131b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.455596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3a56a-2a5e-476a-90ff-41a5e81131b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.455906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3a56a-2a5e-476a-90ff-41a5e81131b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.456335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3a56a-2a5e-476a-90ff-41a5e81131b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.472801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:45.547211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3a56a-2a5e-476a-90ff-41a5e81131b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.548909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3a56a-2a5e-476a-90ff-41a5e81131b1 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.561260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 874/1741] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:45.629772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.631397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.755202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81dfeb-8280-4a2c-aa3d-298f92a473b0 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.762127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.762803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.763070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.763335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] PATH_INFO: `/v3/users/7792c486ebeb4827a01306aeb4f64299` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.764478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7792c486ebeb4827a01306aeb4f64299)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.766863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.818756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7792c486ebeb4827a01306aeb4f64299'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.819561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7792c486ebeb4827a01306aeb4f64299'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.835683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7792c486ebeb4827a01306aeb4f64299'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.847286 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7792c486ebeb4827a01306aeb4f64299'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.848760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7792c486ebeb4827a01306aeb4f64299'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.877674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 868/1742] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:45 2022] DELETE /identity/v3/users/7792c486ebeb4827a01306aeb4f64299 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:45.882589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4e37a-20ac-4ff2-aa82-ab02cc35cc11 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.882878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4e37a-20ac-4ff2-aa82-ab02cc35cc11 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.883184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4e37a-20ac-4ff2-aa82-ab02cc35cc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.905810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-79f4e37a-20ac-4ff2-aa82-ab02cc35cc11 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:45.906819 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-79f4e37a-20ac-4ff2-aa82-ab02cc35cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.988460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 875/1743] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:45.998510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.999928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.055640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:46.062433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fd35-2197-4abb-98e8-147b03d1ccf6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.067811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a6f3c-0993-4a5c-a6b9-e872cb5ec2f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.068430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a6f3c-0993-4a5c-a6b9-e872cb5ec2f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.068732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a6f3c-0993-4a5c-a6b9-e872cb5ec2f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.068989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a6f3c-0993-4a5c-a6b9-e872cb5ec2f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.080020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a6f3c-0993-4a5c-a6b9-e872cb5ec2f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.082577 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a6f3c-0993-4a5c-a6b9-e872cb5ec2f3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.093882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 869/1744] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:46.408858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4e37a-20ac-4ff2-aa82-ab02cc35cc11 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.410573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-79f4e37a-20ac-4ff2-aa82-ab02cc35cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.423280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4e37a-20ac-4ff2-aa82-ab02cc35cc11 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.428713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.429377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.429677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.429952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] PATH_INFO: `/v3/projects/9e392760b67f4f8685f95d64a1e8821b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.431245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e392760b67f4f8685f95d64a1e8821b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.433414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.487765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e392760b67f4f8685f95d64a1e8821b', 'request_id': 'req-32eaae2d-c645-4083-9e61-883efcd250b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.497628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e392760b67f4f8685f95d64a1e8821b', 'request_id': 'req-32eaae2d-c645-4083-9e61-883efcd250b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.504931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e392760b67f4f8685f95d64a1e8821b', 'request_id': 'req-32eaae2d-c645-4083-9e61-883efcd250b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.516209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] The token cache is being invalidate because project 9e392760b67f4f8685f95d64a1e8821b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:46.516501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.517477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-32eaae2d-c645-4083-9e61-883efcd250b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.519133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 876/1745] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:46 2022] DELETE /identity/v3/projects/9e392760b67f4f8685f95d64a1e8821b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:46.523180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a6f3c-0993-4a5c-a6b9-e872cb5ec2f3 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.533463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a6f3c-0993-4a5c-a6b9-e872cb5ec2f3 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.595041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.596113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.596487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.596839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] PATH_INFO: `/v3/users/c2a02b40c76241e28a011e4deab37b14` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.598938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2a02b40c76241e28a011e4deab37b14)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.601311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.652933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2a02b40c76241e28a011e4deab37b14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.653621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2a02b40c76241e28a011e4deab37b14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.676785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2a02b40c76241e28a011e4deab37b14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.686831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2a02b40c76241e28a011e4deab37b14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.687957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2a02b40c76241e28a011e4deab37b14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.701649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 870/1746] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:46 2022] DELETE /identity/v3/users/c2a02b40c76241e28a011e4deab37b14 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:46.791210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df7e3-b6a9-4fb5-91c8-edbda193c84a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.791509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df7e3-b6a9-4fb5-91c8-edbda193c84a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.791763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df7e3-b6a9-4fb5-91c8-edbda193c84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.813685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f66df7e3-b6a9-4fb5-91c8-edbda193c84a None None] MFA Rules not processed for user `1d750b1b27d840b38fd541653a61640b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:46.815206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f66df7e3-b6a9-4fb5-91c8-edbda193c84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.839002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:46.908162 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 877/1747] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:47.069690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.072111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:47.178378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b12652-b0ac-46bb-828b-01870417abc9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.184450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.185318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.185667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.185966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] PATH_INFO: `/v3/projects/c6b481367d1c4a9c8f4b05dce9160d21` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.191124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b481367d1c4a9c8f4b05dce9160d21)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.193259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:47.206719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:47.274271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b481367d1c4a9c8f4b05dce9160d21', 'request_id': 'req-81d2d8ec-2630-457e-925e-ed4e344fc5b7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.283287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b481367d1c4a9c8f4b05dce9160d21', 'request_id': 'req-81d2d8ec-2630-457e-925e-ed4e344fc5b7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.293275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b481367d1c4a9c8f4b05dce9160d21', 'request_id': 'req-81d2d8ec-2630-457e-925e-ed4e344fc5b7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.309347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] The token cache is being invalidate because project c6b481367d1c4a9c8f4b05dce9160d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:47.309781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.310784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.312484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 871/1748] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:47 2022] DELETE /identity/v3/projects/c6b481367d1c4a9c8f4b05dce9160d21 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:47.316578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66df7e3-b6a9-4fb5-91c8-edbda193c84a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.329363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66df7e3-b6a9-4fb5-91c8-edbda193c84a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.398671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.399421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.399795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.400173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] PATH_INFO: `/v3/users/ac65638aa53d4304a1d41179a02b2204` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.401451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac65638aa53d4304a1d41179a02b2204)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.403720 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:47.493461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac65638aa53d4304a1d41179a02b2204'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.497511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac65638aa53d4304a1d41179a02b2204'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.515649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac65638aa53d4304a1d41179a02b2204'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.523677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac65638aa53d4304a1d41179a02b2204'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.524419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-024bae07-ba40-4a63-b581-1006109b6eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac65638aa53d4304a1d41179a02b2204'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.541156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 878/1749] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:47 2022] DELETE /identity/v3/users/ac65638aa53d4304a1d41179a02b2204 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:47.904160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.906004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:47.929165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2d8ec-2630-457e-925e-ed4e344fc5b7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.064809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.066781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.067804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.069045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] PATH_INFO: `/v3/projects/98cba1581ae34d5fb0377798bdacdf8a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.071690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=98cba1581ae34d5fb0377798bdacdf8a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.073853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.237799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98cba1581ae34d5fb0377798bdacdf8a', 'request_id': 'req-cbecd97d-72df-45b1-b875-268895d51e7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.249526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98cba1581ae34d5fb0377798bdacdf8a', 'request_id': 'req-cbecd97d-72df-45b1-b875-268895d51e7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.260103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98cba1581ae34d5fb0377798bdacdf8a', 'request_id': 'req-cbecd97d-72df-45b1-b875-268895d51e7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.275573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] The token cache is being invalidate because project 98cba1581ae34d5fb0377798bdacdf8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:48.275928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.276686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cbecd97d-72df-45b1-b875-268895d51e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.278644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 872/1750] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:47 2022] DELETE /identity/v3/projects/98cba1581ae34d5fb0377798bdacdf8a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:48.283967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605b170-d4aa-4187-9ccb-c3735bc697a4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.284256 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605b170-d4aa-4187-9ccb-c3735bc697a4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.284635 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605b170-d4aa-4187-9ccb-c3735bc697a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.311961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3605b170-d4aa-4187-9ccb-c3735bc697a4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:48.314529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3605b170-d4aa-4187-9ccb-c3735bc697a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.409012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:48.427551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 879/1751] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:48.435160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff69a0-4acd-45fe-9979-f9650b8857d3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.435482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff69a0-4acd-45fe-9979-f9650b8857d3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.435746 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff69a0-4acd-45fe-9979-f9650b8857d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.457230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0dff69a0-4acd-45fe-9979-f9650b8857d3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:48.459346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0dff69a0-4acd-45fe-9979-f9650b8857d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.468212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 873/1752] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:48.472806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605b170-d4aa-4187-9ccb-c3735bc697a4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.491990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605b170-d4aa-4187-9ccb-c3735bc697a4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.503041 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.503950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.504442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.504969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.506128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:48.506476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.508219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.537600 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 880/1753] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:48.541772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff69a0-4acd-45fe-9979-f9650b8857d3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.551793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff69a0-4acd-45fe-9979-f9650b8857d3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.556461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa62fa-f37d-4382-b850-d6948fb608c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.557096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa62fa-f37d-4382-b850-d6948fb608c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.557372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa62fa-f37d-4382-b850-d6948fb608c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.557664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa62fa-f37d-4382-b850-d6948fb608c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.558415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa62fa-f37d-4382-b850-d6948fb608c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.560311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa62fa-f37d-4382-b850-d6948fb608c0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.601959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:48.610926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 874/1754] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:48.614244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.625355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15478394-ee72-41b7-9e19-4305ba0d824b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.692988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65619fd7-5680-461f-82bc-6fd1da06ae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.693743 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65619fd7-5680-461f-82bc-6fd1da06ae43 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.693845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65619fd7-5680-461f-82bc-6fd1da06ae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.694113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65619fd7-5680-461f-82bc-6fd1da06ae43 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.694790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65619fd7-5680-461f-82bc-6fd1da06ae43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.696663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65619fd7-5680-461f-82bc-6fd1da06ae43 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.721883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 881/1755] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:48.726119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa62fa-f37d-4382-b850-d6948fb608c0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.737140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa62fa-f37d-4382-b850-d6948fb608c0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.742076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a5ce-c673-4044-9e97-2a32991b2b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.742702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a5ce-c673-4044-9e97-2a32991b2b16 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.742976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a5ce-c673-4044-9e97-2a32991b2b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.743264 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a5ce-c673-4044-9e97-2a32991b2b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.743968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1a5ce-c673-4044-9e97-2a32991b2b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.745731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1a5ce-c673-4044-9e97-2a32991b2b16 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.762213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 875/1756] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:48.764613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65619fd7-5680-461f-82bc-6fd1da06ae43 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.776246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65619fd7-5680-461f-82bc-6fd1da06ae43 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.780904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a518a-fee1-4ed4-b56c-50420e9ee651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.781829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a518a-fee1-4ed4-b56c-50420e9ee651 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.781953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a518a-fee1-4ed4-b56c-50420e9ee651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.782258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a518a-fee1-4ed4-b56c-50420e9ee651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.783059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a518a-fee1-4ed4-b56c-50420e9ee651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.785035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a518a-fee1-4ed4-b56c-50420e9ee651 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.800701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 882/1757] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:48.804889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a5ce-c673-4044-9e97-2a32991b2b16 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.815521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a5ce-c673-4044-9e97-2a32991b2b16 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.820539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.821295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.821637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.821976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] PATH_INFO: `/v3/projects/c46520aae62641dfbd846320a29285fe/users/3ad3dfc91484458596870ce8fe2ed89d/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.838441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46520aae62641dfbd846320a29285fe, user_id=3ad3dfc91484458596870ce8fe2ed89d, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.841180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.862554 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 876/1758] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:48 2022] PUT /identity/v3/projects/c46520aae62641dfbd846320a29285fe/users/3ad3dfc91484458596870ce8fe2ed89d/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:48.873667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acb4c5-1319-4f1a-a118-3eab9134aa62 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.873931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acb4c5-1319-4f1a-a118-3eab9134aa62 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.874264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acb4c5-1319-4f1a-a118-3eab9134aa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.916905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-33acb4c5-1319-4f1a-a118-3eab9134aa62 None None] MFA Rules not processed for user `3ad3dfc91484458596870ce8fe2ed89d`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:48.918177 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-33acb4c5-1319-4f1a-a118-3eab9134aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.991070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 883/1759] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:49.002397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.004307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:49.101924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc80a67-da9b-4661-a32a-4baab4ac5cb3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.106531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4bf3b-ff0d-47b1-8875-376c6d8f17f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.107358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4bf3b-ff0d-47b1-8875-376c6d8f17f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.107832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4bf3b-ff0d-47b1-8875-376c6d8f17f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.108299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4bf3b-ff0d-47b1-8875-376c6d8f17f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.119932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4bf3b-ff0d-47b1-8875-376c6d8f17f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.121603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4bf3b-ff0d-47b1-8875-376c6d8f17f1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.155585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 877/1760] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:52.300196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acb4c5-1319-4f1a-a118-3eab9134aa62 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:52.301882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-33acb4c5-1319-4f1a-a118-3eab9134aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:52.390884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:52.424087 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acb4c5-1319-4f1a-a118-3eab9134aa62 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:52.429729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:52.430642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:52.431176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:52.431652 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] PATH_INFO: `/v3/users/5edb902b5d464592ae405adf19c0f358` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:52.433270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5edb902b5d464592ae405adf19c0f358)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:52.435356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:52.529947 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5edb902b5d464592ae405adf19c0f358'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.530710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5edb902b5d464592ae405adf19c0f358'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.543630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5edb902b5d464592ae405adf19c0f358'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.550745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5edb902b5d464592ae405adf19c0f358'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.551469 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5edb902b5d464592ae405adf19c0f358'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.570139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 884/1761] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:52 2022] DELETE /identity/v3/users/5edb902b5d464592ae405adf19c0f358 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:52.573611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4489c3-08c3-4c21-a17a-4bcda687188d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:52.575039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4489c3-08c3-4c21-a17a-4bcda687188d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:52.575581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4489c3-08c3-4c21-a17a-4bcda687188d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:52.613670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7d4489c3-08c3-4c21-a17a-4bcda687188d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:52.615050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d4489c3-08c3-4c21-a17a-4bcda687188d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:52.687182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 878/1762] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:52.694335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:52.696391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:52.850045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191edb5-e9f5-4ce3-88b9-6f2e46a2731c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:52.869729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47c8a7-dd7a-4bcb-bbb6-d6a424850396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:52.876566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47c8a7-dd7a-4bcb-bbb6-d6a424850396 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:52.876859 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47c8a7-dd7a-4bcb-bbb6-d6a424850396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:52.877138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47c8a7-dd7a-4bcb-bbb6-d6a424850396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:52.897682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47c8a7-dd7a-4bcb-bbb6-d6a424850396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:52.899185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47c8a7-dd7a-4bcb-bbb6-d6a424850396 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:52.914434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 885/1763] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:53.518352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4489c3-08c3-4c21-a17a-4bcda687188d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.521394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d4489c3-08c3-4c21-a17a-4bcda687188d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:53.537234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4489c3-08c3-4c21-a17a-4bcda687188d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.542369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.543301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.543666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.544126 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] PATH_INFO: `/v3/projects/8ff207dc6bef49c68ab81eaf90ae71a9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.545499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff207dc6bef49c68ab81eaf90ae71a9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.548921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.623592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff207dc6bef49c68ab81eaf90ae71a9', 'request_id': 'req-ef0380d8-032b-47c9-8074-9120a6738675'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.630065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff207dc6bef49c68ab81eaf90ae71a9', 'request_id': 'req-ef0380d8-032b-47c9-8074-9120a6738675'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.639356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff207dc6bef49c68ab81eaf90ae71a9', 'request_id': 'req-ef0380d8-032b-47c9-8074-9120a6738675'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.642875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:53.654890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] The token cache is being invalidate because project 8ff207dc6bef49c68ab81eaf90ae71a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:53.655254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.656119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.661800 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 879/1764] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:53 2022] DELETE /identity/v3/projects/8ff207dc6bef49c68ab81eaf90ae71a9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:53.662970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47c8a7-dd7a-4bcb-bbb6-d6a424850396 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.672589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47c8a7-dd7a-4bcb-bbb6-d6a424850396 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.736304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.737200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.737549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.737901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] PATH_INFO: `/v3/users/badb315377134771a62a9b02ad18ce37` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.739588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=badb315377134771a62a9b02ad18ce37)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.741626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.755534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:53.826256 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badb315377134771a62a9b02ad18ce37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.827216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badb315377134771a62a9b02ad18ce37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.841904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badb315377134771a62a9b02ad18ce37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.852688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badb315377134771a62a9b02ad18ce37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.854253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fb1bf33e-ccb6-4d10-a901-35cf8851aa29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badb315377134771a62a9b02ad18ce37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:53.871449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 886/1765] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:53 2022] DELETE /identity/v3/users/badb315377134771a62a9b02ad18ce37 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:54.285977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:54.293662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:54.361374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0380d8-032b-47c9-8074-9120a6738675 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:54.565911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:54.566678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:54.569566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:54.570065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] PATH_INFO: `/v3/projects/947b45c0e5024703b4fe4b8579be4c21` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:54.572599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=947b45c0e5024703b4fe4b8579be4c21)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:54.576661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:54.744371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947b45c0e5024703b4fe4b8579be4c21', 'request_id': 'req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.759982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947b45c0e5024703b4fe4b8579be4c21', 'request_id': 'req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.777194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947b45c0e5024703b4fe4b8579be4c21', 'request_id': 'req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.799990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] The token cache is being invalidate because project 947b45c0e5024703b4fe4b8579be4c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:54.800509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.801352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b3a855c-f7e0-496a-a6d5-0eecebd419c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.803314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 880/1766] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:54 2022] DELETE /identity/v3/projects/947b45c0e5024703b4fe4b8579be4c21 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:54.807906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90de4-4bbe-41b4-8fb0-bd1e7bb29f9e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:54.808858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90de4-4bbe-41b4-8fb0-bd1e7bb29f9e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:54.809283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90de4-4bbe-41b4-8fb0-bd1e7bb29f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:54.843721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bdc90de4-4bbe-41b4-8fb0-bd1e7bb29f9e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:54.845562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bdc90de4-4bbe-41b4-8fb0-bd1e7bb29f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:55.017612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11366c-72e3-46ef-9c39-dbfa778ca846 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.017900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11366c-72e3-46ef-9c39-dbfa778ca846 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.018200 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11366c-72e3-46ef-9c39-dbfa778ca846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.021423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 887/1767] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:55.040768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8b11366c-72e3-46ef-9c39-dbfa778ca846 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:55.042160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8b11366c-72e3-46ef-9c39-dbfa778ca846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:55.058142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 881/1768] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:55.062108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc90de4-4bbe-41b4-8fb0-bd1e7bb29f9e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.072841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc90de4-4bbe-41b4-8fb0-bd1e7bb29f9e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.078423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.079275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.079425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.080030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.080232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:55.080538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.082232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.109712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11366c-72e3-46ef-9c39-dbfa778ca846 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.115994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 888/1769] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:55.121120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11366c-72e3-46ef-9c39-dbfa778ca846 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.127262 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cc4cd-5b08-4fa9-93de-2f1536efcf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.127961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cc4cd-5b08-4fa9-93de-2f1536efcf7d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.128308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cc4cd-5b08-4fa9-93de-2f1536efcf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.129428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cc4cd-5b08-4fa9-93de-2f1536efcf7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.130171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cc4cd-5b08-4fa9-93de-2f1536efcf7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.132565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cc4cd-5b08-4fa9-93de-2f1536efcf7d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.169264 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 882/1770] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:55 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:55.173504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.185160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730eee8-ddbf-4dcb-85a3-9254389af33b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.262701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5055cc-5130-4851-87e6-fab580fe2302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.263332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5055cc-5130-4851-87e6-fab580fe2302 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.263611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5055cc-5130-4851-87e6-fab580fe2302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.263886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5055cc-5130-4851-87e6-fab580fe2302 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.264523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5055cc-5130-4851-87e6-fab580fe2302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.266825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5055cc-5130-4851-87e6-fab580fe2302 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.314937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 889/1771] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:55 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:55.319235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cc4cd-5b08-4fa9-93de-2f1536efcf7d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.345491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cc4cd-5b08-4fa9-93de-2f1536efcf7d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.364822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc5ae9-f637-413a-8d37-6e0de5ead8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.366289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc5ae9-f637-413a-8d37-6e0de5ead8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.366702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc5ae9-f637-413a-8d37-6e0de5ead8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.366980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc5ae9-f637-413a-8d37-6e0de5ead8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.367644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc5ae9-f637-413a-8d37-6e0de5ead8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.369346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc5ae9-f637-413a-8d37-6e0de5ead8fe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.393612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 883/1772] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:55.397738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5055cc-5130-4851-87e6-fab580fe2302 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.423203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5055cc-5130-4851-87e6-fab580fe2302 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.428543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f68c84-0026-482f-ac2f-b243ee52c759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.429169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68c84-0026-482f-ac2f-b243ee52c759 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.429454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68c84-0026-482f-ac2f-b243ee52c759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.429731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68c84-0026-482f-ac2f-b243ee52c759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.430473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f68c84-0026-482f-ac2f-b243ee52c759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.432421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f68c84-0026-482f-ac2f-b243ee52c759 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.434206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:55.453115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc5ae9-f637-413a-8d37-6e0de5ead8fe admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.453367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 890/1773] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:55.464958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc5ae9-f637-413a-8d37-6e0de5ead8fe admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.472394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.473661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.473661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.473661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] PATH_INFO: `/v3/projects/eba20979bcb947d3b4266f15b0d8f49f/users/b2e5887f216d449291bd91be8a8a37b9/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.499500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba20979bcb947d3b4266f15b0d8f49f, user_id=b2e5887f216d449291bd91be8a8a37b9, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.502178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.524633 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 884/1774] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:55 2022] PUT /identity/v3/projects/eba20979bcb947d3b4266f15b0d8f49f/users/b2e5887f216d449291bd91be8a8a37b9/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:55.526767 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c5f68-77ae-412a-9730-feeb97d00bac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.527054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c5f68-77ae-412a-9730-feeb97d00bac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.527378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c5f68-77ae-412a-9730-feeb97d00bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.568299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ee0c5f68-77ae-412a-9730-feeb97d00bac None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:55.569536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ee0c5f68-77ae-412a-9730-feeb97d00bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:55.645583 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 891/1775] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:55.656965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.658797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:55.684900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:55.734302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a5c59-f301-427e-8e70-a31aa10bded9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.738866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05dd806-9421-4f91-a346-fc3c4606f5f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.739592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dd806-9421-4f91-a346-fc3c4606f5f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.739878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dd806-9421-4f91-a346-fc3c4606f5f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.740152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dd806-9421-4f91-a346-fc3c4606f5f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.753864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05dd806-9421-4f91-a346-fc3c4606f5f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.756303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05dd806-9421-4f91-a346-fc3c4606f5f2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.778622 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 885/1776] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:56.603086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479795e4-9af3-4587-a22a-2becf7389953 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.603631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479795e4-9af3-4587-a22a-2becf7389953 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.603631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479795e4-9af3-4587-a22a-2becf7389953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.648860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-479795e4-9af3-4587-a22a-2becf7389953 None None] MFA Rules not processed for user `b2e5887f216d449291bd91be8a8a37b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:56.650285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-479795e4-9af3-4587-a22a-2becf7389953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:56.790841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 892/1777] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:56.792240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05dd806-9421-4f91-a346-fc3c4606f5f2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.803479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05dd806-9421-4f91-a346-fc3c4606f5f2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.812389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931570bb-e084-40b3-a9b9-c64c5907b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.813568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931570bb-e084-40b3-a9b9-c64c5907b961 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.813898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931570bb-e084-40b3-a9b9-c64c5907b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.814269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931570bb-e084-40b3-a9b9-c64c5907b961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.814918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931570bb-e084-40b3-a9b9-c64c5907b961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.817053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931570bb-e084-40b3-a9b9-c64c5907b961 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.865952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 886/1778] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:56 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:56.867624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479795e4-9af3-4587-a22a-2becf7389953 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.884874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479795e4-9af3-4587-a22a-2becf7389953 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.983719 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaf169-f5b6-4fab-b2e4-6cf1ffefae85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.984805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaf169-f5b6-4fab-b2e4-6cf1ffefae85 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.985224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaf169-f5b6-4fab-b2e4-6cf1ffefae85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.985651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaf169-f5b6-4fab-b2e4-6cf1ffefae85 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.986598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daaf169-f5b6-4fab-b2e4-6cf1ffefae85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.989258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daaf169-f5b6-4fab-b2e4-6cf1ffefae85 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:57.015325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 893/1779] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:56 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:57.019231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931570bb-e084-40b3-a9b9-c64c5907b961 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:57.030887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931570bb-e084-40b3-a9b9-c64c5907b961 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:57.038478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256ed64-6b51-472b-92f1-bb97332fbbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:57.039229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256ed64-6b51-472b-92f1-bb97332fbbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:57.039605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256ed64-6b51-472b-92f1-bb97332fbbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:57.039975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256ed64-6b51-472b-92f1-bb97332fbbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:57.040721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256ed64-6b51-472b-92f1-bb97332fbbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:57.042598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256ed64-6b51-472b-92f1-bb97332fbbb7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:57.056564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 887/1780] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:57.060528 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaf169-f5b6-4fab-b2e4-6cf1ffefae85 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:57.070563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaf169-f5b6-4fab-b2e4-6cf1ffefae85 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:57.075439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7b268-3ed1-490a-84f2-32c4cce9ff1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:57.076318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7b268-3ed1-490a-84f2-32c4cce9ff1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:57.076694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7b268-3ed1-490a-84f2-32c4cce9ff1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:57.077062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7b268-3ed1-490a-84f2-32c4cce9ff1c admin admin] PATH_INFO: `/v3/projects/f12bb1f66c3e4a8c8eedb879d5433b9a/users/d9585879a9c34bffbf1c6813329b1824/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:57.097629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:57.100624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7b268-3ed1-490a-84f2-32c4cce9ff1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12bb1f66c3e4a8c8eedb879d5433b9a, user_id=d9585879a9c34bffbf1c6813329b1824, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:57.103606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7b268-3ed1-490a-84f2-32c4cce9ff1c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:57.119519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 894/1781] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:57 2022] PUT /identity/v3/projects/f12bb1f66c3e4a8c8eedb879d5433b9a/users/d9585879a9c34bffbf1c6813329b1824/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:57.124711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256ed64-6b51-472b-92f1-bb97332fbbb7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:57.140262 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256ed64-6b51-472b-92f1-bb97332fbbb7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:57.209090 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8de58e-3142-42a9-84ae-0693b4e6755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:57.209717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8de58e-3142-42a9-84ae-0693b4e6755d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:57.209993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8de58e-3142-42a9-84ae-0693b4e6755d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:57.210312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8de58e-3142-42a9-84ae-0693b4e6755d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:57.210954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8de58e-3142-42a9-84ae-0693b4e6755d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:57.212550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8de58e-3142-42a9-84ae-0693b4e6755d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:57.224207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 888/1782] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:57.228367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7b268-3ed1-490a-84f2-32c4cce9ff1c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:57.246178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7b268-3ed1-490a-84f2-32c4cce9ff1c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:57.251127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:57.251792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:57.252065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:57.252335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] PATH_INFO: `/v3/projects/f12bb1f66c3e4a8c8eedb879d5433b9a/users/d9585879a9c34bffbf1c6813329b1824/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:57.254373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12bb1f66c3e4a8c8eedb879d5433b9a, user_id=d9585879a9c34bffbf1c6813329b1824, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:57.256901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:57.272738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 895/1783] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:57 2022] PUT /identity/v3/projects/f12bb1f66c3e4a8c8eedb879d5433b9a/users/d9585879a9c34bffbf1c6813329b1824/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:58.118500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ffff3-118a-403f-bded-f0b4dbca829b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:58.119408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ffff3-118a-403f-bded-f0b4dbca829b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:58.119848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ffff3-118a-403f-bded-f0b4dbca829b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:58.168632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a85ffff3-118a-403f-bded-f0b4dbca829b None None] MFA Rules not processed for user `d9585879a9c34bffbf1c6813329b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:58.170756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a85ffff3-118a-403f-bded-f0b4dbca829b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:58.285131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 889/1784] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:28:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:58.300296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:58.302340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:58.412675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddba526-16fe-4204-a47f-3d7344a25849 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:58.416966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:58.417689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:58.418029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:58.418497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:58.490594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:58.492393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:58.502275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 896/1785] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:00.994054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ffff3-118a-403f-bded-f0b4dbca829b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:00.995618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a85ffff3-118a-403f-bded-f0b4dbca829b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:01.008864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ffff3-118a-403f-bded-f0b4dbca829b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:01.013368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7e3a3-1ce0-4eab-bd6f-f5c154098d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:01.013964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7e3a3-1ce0-4eab-bd6f-f5c154098d68 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:01.014300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7e3a3-1ce0-4eab-bd6f-f5c154098d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:01.014572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7e3a3-1ce0-4eab-bd6f-f5c154098d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:01.025141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:01.027898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7e3a3-1ce0-4eab-bd6f-f5c154098d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:01.030000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7e3a3-1ce0-4eab-bd6f-f5c154098d68 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:01.043344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 890/1786] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:02.320268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:02.321702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:02.391521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b47b2-ee0d-4825-831a-ac0f239e734e service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:02.396269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:02.396868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:02.397141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:02.397442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] PATH_INFO: `/v3/users/3ad3dfc91484458596870ce8fe2ed89d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:02.398648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad3dfc91484458596870ce8fe2ed89d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:02.400539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:02.440672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:02.451778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad3dfc91484458596870ce8fe2ed89d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:02.452541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad3dfc91484458596870ce8fe2ed89d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:02.465645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad3dfc91484458596870ce8fe2ed89d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:02.473936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad3dfc91484458596870ce8fe2ed89d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:02.474708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad3dfc91484458596870ce8fe2ed89d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:02.488271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 897/1787] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:02 2022] DELETE /identity/v3/users/3ad3dfc91484458596870ce8fe2ed89d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:02.493486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0cce-0d11-4909-a305-e0fc42ca3726 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:02.494325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0cce-0d11-4909-a305-e0fc42ca3726 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:02.494609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb0cce-0d11-4909-a305-e0fc42ca3726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:02.515072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c9fb0cce-0d11-4909-a305-e0fc42ca3726 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:02.516411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9fb0cce-0d11-4909-a305-e0fc42ca3726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:02.584964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 891/1788] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:02.595322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:02.596869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:02.657222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa3bd-ab39-4c81-9c7a-b9beeb34f2f7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:02.661111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:02.661716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:02.661990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:02.662285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:02.672112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:02.673549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:02.683530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 898/1789] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:02.956821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb0cce-0d11-4909-a305-e0fc42ca3726 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:02.958307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9fb0cce-0d11-4909-a305-e0fc42ca3726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:02.971094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb0cce-0d11-4909-a305-e0fc42ca3726 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:02.976151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:02.976888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:02.977229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:02.977598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] PATH_INFO: `/v3/projects/c46520aae62641dfbd846320a29285fe` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:02.978929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46520aae62641dfbd846320a29285fe)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:02.981039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:03.028604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46520aae62641dfbd846320a29285fe', 'request_id': 'req-bf21a507-a503-4bd1-bb39-376152efb26f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:03.035054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46520aae62641dfbd846320a29285fe', 'request_id': 'req-bf21a507-a503-4bd1-bb39-376152efb26f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:03.043955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46520aae62641dfbd846320a29285fe', 'request_id': 'req-bf21a507-a503-4bd1-bb39-376152efb26f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:03.056552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] The token cache is being invalidate because project c46520aae62641dfbd846320a29285fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:03.056841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:03.057478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:03.059014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 892/1790] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:02 2022] DELETE /identity/v3/projects/c46520aae62641dfbd846320a29285fe => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:12.678946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=42,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:13.053799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:19.907736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:19.909217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:19.923019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63dbe0-8a18-4c65-b691-b01c28cb242c service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:19.981913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:19.982569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:19.982845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:19.983114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] PATH_INFO: `/v3/users/b2e5887f216d449291bd91be8a8a37b9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:19.984329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e5887f216d449291bd91be8a8a37b9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:19.986268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.036159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e5887f216d449291bd91be8a8a37b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.036723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e5887f216d449291bd91be8a8a37b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.049302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e5887f216d449291bd91be8a8a37b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.056279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e5887f216d449291bd91be8a8a37b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.056940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e5887f216d449291bd91be8a8a37b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.070331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 899/1791] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:19 2022] DELETE /identity/v3/users/b2e5887f216d449291bd91be8a8a37b9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:20.247787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.249302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.262815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21a507-a503-4bd1-bb39-376152efb26f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:20.319794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:20.320340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.320617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.320884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] PATH_INFO: `/v3/projects/eba20979bcb947d3b4266f15b0d8f49f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.322012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eba20979bcb947d3b4266f15b0d8f49f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:20.323939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.372276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eba20979bcb947d3b4266f15b0d8f49f', 'request_id': 'req-a26171f9-a85c-49ac-92a7-a190a159c08c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.378781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eba20979bcb947d3b4266f15b0d8f49f', 'request_id': 'req-a26171f9-a85c-49ac-92a7-a190a159c08c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.391886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eba20979bcb947d3b4266f15b0d8f49f', 'request_id': 'req-a26171f9-a85c-49ac-92a7-a190a159c08c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.403936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] The token cache is being invalidate because project eba20979bcb947d3b4266f15b0d8f49f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:20.404220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.404843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.406428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 893/1792] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:20 2022] DELETE /identity/v3/projects/eba20979bcb947d3b4266f15b0d8f49f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:20.410656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.412155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.425566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ed896-2fbd-4176-90ab-cf8c5504724f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:20.441011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.442569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.492080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:20.492719 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.493128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.493404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] PATH_INFO: `/v3/users/d9585879a9c34bffbf1c6813329b1824` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.495170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9585879a9c34bffbf1c6813329b1824)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:20.498714 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.527543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26171f9-a85c-49ac-92a7-a190a159c08c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:20.532774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:20.533271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:20.533469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.533744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.534025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] PATH_INFO: `/v3/users/fe23b30102114521b7bc797c7a57fd28` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.536072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe23b30102114521b7bc797c7a57fd28)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:20.542531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.572525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9585879a9c34bffbf1c6813329b1824'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.573210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9585879a9c34bffbf1c6813329b1824'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.585360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:20.588864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9585879a9c34bffbf1c6813329b1824'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.598240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9585879a9c34bffbf1c6813329b1824'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.599124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-abe2924f-f456-4df3-8b6a-e51c3809db6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9585879a9c34bffbf1c6813329b1824'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.601537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe23b30102114521b7bc797c7a57fd28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.602247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe23b30102114521b7bc797c7a57fd28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.617302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe23b30102114521b7bc797c7a57fd28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.622359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 900/1793] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:20 2022] DELETE /identity/v3/users/d9585879a9c34bffbf1c6813329b1824 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:20.625017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe23b30102114521b7bc797c7a57fd28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.625761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe23b30102114521b7bc797c7a57fd28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.642130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 894/1794] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:20 2022] DELETE /identity/v3/users/fe23b30102114521b7bc797c7a57fd28 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:20.646793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36c64-1651-405b-bf6c-69a86645b8b8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.647077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36c64-1651-405b-bf6c-69a86645b8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.647358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e36c64-1651-405b-bf6c-69a86645b8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.667286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b2e36c64-1651-405b-bf6c-69a86645b8b8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:20.668717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b2e36c64-1651-405b-bf6c-69a86645b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.746944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 901/1795] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:20.763368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.766037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.844927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a333d-5cf3-4950-814f-df1749e288fa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:20.849744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb80b4c-74b6-4b9a-885d-732c84e2c8b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:20.850744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80b4c-74b6-4b9a-885d-732c84e2c8b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.851026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80b4c-74b6-4b9a-885d-732c84e2c8b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.851490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80b4c-74b6-4b9a-885d-732c84e2c8b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.862683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb80b4c-74b6-4b9a-885d-732c84e2c8b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:20.864342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb80b4c-74b6-4b9a-885d-732c84e2c8b7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.865472 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e36c64-1651-405b-bf6c-69a86645b8b8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.867076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b2e36c64-1651-405b-bf6c-69a86645b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.876474 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 895/1796] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:20.885044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e36c64-1651-405b-bf6c-69a86645b8b8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:20.893323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:20.894149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.894508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.894878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] PATH_INFO: `/v3/projects/f12bb1f66c3e4a8c8eedb879d5433b9a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.896617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12bb1f66c3e4a8c8eedb879d5433b9a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:20.898989 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.964608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12bb1f66c3e4a8c8eedb879d5433b9a', 'request_id': 'req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.976661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12bb1f66c3e4a8c8eedb879d5433b9a', 'request_id': 'req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.987389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12bb1f66c3e4a8c8eedb879d5433b9a', 'request_id': 'req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.004066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] The token cache is being invalidate because project f12bb1f66c3e4a8c8eedb879d5433b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:21.004701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.005679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5d1993d9-7181-4400-a7bc-ec6ac3bc165e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.007567 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 902/1797] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:20 2022] DELETE /identity/v3/projects/f12bb1f66c3e4a8c8eedb879d5433b9a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:21.012690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc103e-0fa9-49b5-8e5e-5ccee15e2b35 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.012973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc103e-0fa9-49b5-8e5e-5ccee15e2b35 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.013247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc103e-0fa9-49b5-8e5e-5ccee15e2b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.036039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-64dc103e-0fa9-49b5-8e5e-5ccee15e2b35 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:21.037568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-64dc103e-0fa9-49b5-8e5e-5ccee15e2b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.116808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 896/1798] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:21.127036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23273b-57d7-4420-bc39-fae8bbc848e0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.127408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23273b-57d7-4420-bc39-fae8bbc848e0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.127697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23273b-57d7-4420-bc39-fae8bbc848e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.157663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1a23273b-57d7-4420-bc39-fae8bbc848e0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:21.158504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1a23273b-57d7-4420-bc39-fae8bbc848e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.167050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 903/1799] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:21.171547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc103e-0fa9-49b5-8e5e-5ccee15e2b35 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.184138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc103e-0fa9-49b5-8e5e-5ccee15e2b35 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.191686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.192381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.193369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.193369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.194237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:21.194553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.196474 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.214230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 897/1800] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:29:21.216507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23273b-57d7-4420-bc39-fae8bbc848e0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.231309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23273b-57d7-4420-bc39-fae8bbc848e0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.239980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.240592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.241055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.241522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.242608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.245355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.261991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:21.288640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 904/1801] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:21.292924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.309360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.309844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a90c4-16a7-45a7-ac55-feca075e2741 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.311119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.371814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:21.402213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522cc01-f480-46a4-a6d0-d152a66a84c6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.402388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1bc86-66c1-47fb-b9e7-19137855e5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.402388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1bc86-66c1-47fb-b9e7-19137855e5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.402388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1bc86-66c1-47fb-b9e7-19137855e5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.402388 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1bc86-66c1-47fb-b9e7-19137855e5c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.402725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1bc86-66c1-47fb-b9e7-19137855e5c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.402725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1bc86-66c1-47fb-b9e7-19137855e5c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.402898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.402898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.402898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.402898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] PATH_INFO: `/v3/projects/b10aabcbed534f298d66ac23b519b1f9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.403186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10aabcbed534f298d66ac23b519b1f9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.404093 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.425402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 898/1802] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:21.426838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1bc86-66c1-47fb-b9e7-19137855e5c6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.436504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1bc86-66c1-47fb-b9e7-19137855e5c6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.441962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e879e5-e3d6-4d3a-8a47-7a2fcd875c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.442657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e879e5-e3d6-4d3a-8a47-7a2fcd875c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.443025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e879e5-e3d6-4d3a-8a47-7a2fcd875c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.443302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e879e5-e3d6-4d3a-8a47-7a2fcd875c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.443944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e879e5-e3d6-4d3a-8a47-7a2fcd875c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.445543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e879e5-e3d6-4d3a-8a47-7a2fcd875c4b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.455620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 899/1803] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:21.457921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10aabcbed534f298d66ac23b519b1f9', 'request_id': 'req-dc1cc839-c341-4ace-be1d-31db049f1151'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.459578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e879e5-e3d6-4d3a-8a47-7a2fcd875c4b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.466066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10aabcbed534f298d66ac23b519b1f9', 'request_id': 'req-dc1cc839-c341-4ace-be1d-31db049f1151'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.469875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e879e5-e3d6-4d3a-8a47-7a2fcd875c4b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.473505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10aabcbed534f298d66ac23b519b1f9', 'request_id': 'req-dc1cc839-c341-4ace-be1d-31db049f1151'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.485204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] The token cache is being invalidate because project b10aabcbed534f298d66ac23b519b1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:21.485494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.486203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc1cc839-c341-4ace-be1d-31db049f1151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.487787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 905/1804] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:21 2022] DELETE /identity/v3/projects/b10aabcbed534f298d66ac23b519b1f9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:21.492425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81349b-c3e2-45d0-b612-5aba5a23f78e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.492713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81349b-c3e2-45d0-b612-5aba5a23f78e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.492990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81349b-c3e2-45d0-b612-5aba5a23f78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.513718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9f81349b-c3e2-45d0-b612-5aba5a23f78e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:21.514973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9f81349b-c3e2-45d0-b612-5aba5a23f78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.530449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.531383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.531844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.532120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.532769 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.534395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.544466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 900/1805] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:21.549456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.551457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.564320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ad5c-2910-4613-a1ae-f56d348d8155 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.569057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459595b1-90e1-4595-9255-efef99cb8ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.569695 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459595b1-90e1-4595-9255-efef99cb8ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.569981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459595b1-90e1-4595-9255-efef99cb8ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.570278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459595b1-90e1-4595-9255-efef99cb8ccd admin admin] PATH_INFO: `/v3/projects/de61e70413c34586a8fa98b33ea252e0/users/5c15cd57ca6843868883d4f081e0725c/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.576623 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 906/1806] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:21.584197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce75a7-5983-4454-b221-1e7b7e1d21b1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.584481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce75a7-5983-4454-b221-1e7b7e1d21b1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.584756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce75a7-5983-4454-b221-1e7b7e1d21b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.585717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459595b1-90e1-4595-9255-efef99cb8ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=de61e70413c34586a8fa98b33ea252e0, user_id=5c15cd57ca6843868883d4f081e0725c, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.588517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459595b1-90e1-4595-9255-efef99cb8ccd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.602889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 901/1807] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:21 2022] PUT /identity/v3/projects/de61e70413c34586a8fa98b33ea252e0/users/5c15cd57ca6843868883d4f081e0725c/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:21.609033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8cce75a7-5983-4454-b221-1e7b7e1d21b1 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:21.610783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cce75a7-5983-4454-b221-1e7b7e1d21b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.616378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632a4b2-b366-479f-96a8-8df5bf99c8a5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.616656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632a4b2-b366-479f-96a8-8df5bf99c8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.616918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632a4b2-b366-479f-96a8-8df5bf99c8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.643812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:21.648335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e632a4b2-b366-479f-96a8-8df5bf99c8a5 None None] MFA Rules not processed for user `5c15cd57ca6843868883d4f081e0725c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:21.649615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e632a4b2-b366-479f-96a8-8df5bf99c8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.688392 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 907/1808] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:21.692631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce75a7-5983-4454-b221-1e7b7e1d21b1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.702650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce75a7-5983-4454-b221-1e7b7e1d21b1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.707478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.708088 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.708368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.708643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.709291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:21.709599 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.712039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.717269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 902/1809] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:21.728583 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 908/1810] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:29:21.728839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632a4b2-b366-479f-96a8-8df5bf99c8a5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.730233 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e632a4b2-b366-479f-96a8-8df5bf99c8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.732578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.742509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21748658-1423-4e94-a953-93d7ec1d2de1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.747156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c6f54-951f-4cfb-b9e5-93ab28e8a6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.747749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6f54-951f-4cfb-b9e5-93ab28e8a6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.748027 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6f54-951f-4cfb-b9e5-93ab28e8a6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.748309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6f54-951f-4cfb-b9e5-93ab28e8a6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.748932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c6f54-951f-4cfb-b9e5-93ab28e8a6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.751066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c6f54-951f-4cfb-b9e5-93ab28e8a6e2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.752012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:21.783532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 909/1811] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:21.787615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c6f54-951f-4cfb-b9e5-93ab28e8a6e2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.792229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632a4b2-b366-479f-96a8-8df5bf99c8a5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.841089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaf5bb-a712-4ceb-ad91-0ff2f94db4c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.841697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaf5bb-a712-4ceb-ad91-0ff2f94db4c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.841975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaf5bb-a712-4ceb-ad91-0ff2f94db4c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.842278 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaf5bb-a712-4ceb-ad91-0ff2f94db4c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.850528 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c6f54-951f-4cfb-b9e5-93ab28e8a6e2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.855813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597bc3b-7c8b-428f-b2ee-046ffb0faee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.856457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597bc3b-7c8b-428f-b2ee-046ffb0faee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.856762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597bc3b-7c8b-428f-b2ee-046ffb0faee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.857034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597bc3b-7c8b-428f-b2ee-046ffb0faee9 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.857668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597bc3b-7c8b-428f-b2ee-046ffb0faee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.859858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597bc3b-7c8b-428f-b2ee-046ffb0faee9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.883201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 910/1812] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:21.887279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597bc3b-7c8b-428f-b2ee-046ffb0faee9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.897383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597bc3b-7c8b-428f-b2ee-046ffb0faee9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.901270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaf5bb-a712-4ceb-ad91-0ff2f94db4c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.902766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaf5bb-a712-4ceb-ad91-0ff2f94db4c2 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.905068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1e6d3-1f0f-41b7-8fb9-3b49f596625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.905667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1e6d3-1f0f-41b7-8fb9-3b49f596625d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.905942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1e6d3-1f0f-41b7-8fb9-3b49f596625d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.906297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1e6d3-1f0f-41b7-8fb9-3b49f596625d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.907131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1e6d3-1f0f-41b7-8fb9-3b49f596625d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.908730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1e6d3-1f0f-41b7-8fb9-3b49f596625d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.921592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 903/1813] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:21.924914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 911/1814] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:21.925987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaf5bb-a712-4ceb-ad91-0ff2f94db4c2 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.937488 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaf5bb-a712-4ceb-ad91-0ff2f94db4c2 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.944098 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1e6d3-1f0f-41b7-8fb9-3b49f596625d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.947201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f113f3-9aee-4456-96b6-8ce38764e350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.947912 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f113f3-9aee-4456-96b6-8ce38764e350 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.948256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f113f3-9aee-4456-96b6-8ce38764e350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.948589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f113f3-9aee-4456-96b6-8ce38764e350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.949294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f113f3-9aee-4456-96b6-8ce38764e350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.951929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f113f3-9aee-4456-96b6-8ce38764e350 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.955208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1e6d3-1f0f-41b7-8fb9-3b49f596625d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.960605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.961591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.961866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.962157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] PATH_INFO: `/v3/users/5c15cd57ca6843868883d4f081e0725c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.963176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 904/1815] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:21.966379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c15cd57ca6843868883d4f081e0725c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.967252 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f113f3-9aee-4456-96b6-8ce38764e350 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.968574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.978925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f113f3-9aee-4456-96b6-8ce38764e350 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.983543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7028a0-40ac-44e7-a3ed-dec687760876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.984417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7028a0-40ac-44e7-a3ed-dec687760876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.984903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7028a0-40ac-44e7-a3ed-dec687760876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.985277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7028a0-40ac-44e7-a3ed-dec687760876 admin admin] PATH_INFO: `/v3/projects/8f6ae36207a4410e94904ec198c0b1c2/users/f9168731dea04eb5b423569f846ab0a6/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.003471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7028a0-40ac-44e7-a3ed-dec687760876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6ae36207a4410e94904ec198c0b1c2, user_id=f9168731dea04eb5b423569f846ab0a6, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.006551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7028a0-40ac-44e7-a3ed-dec687760876 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.020478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 905/1816] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:21 2022] PUT /identity/v3/projects/8f6ae36207a4410e94904ec198c0b1c2/users/f9168731dea04eb5b423569f846ab0a6/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:22.024029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c15cd57ca6843868883d4f081e0725c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.024709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c15cd57ca6843868883d4f081e0725c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.031544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b09d3-f197-43b7-96b9-33d2ea26c8a6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.031915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b09d3-f197-43b7-96b9-33d2ea26c8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.032232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b09d3-f197-43b7-96b9-33d2ea26c8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.036209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:22.037739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c15cd57ca6843868883d4f081e0725c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.046068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c15cd57ca6843868883d4f081e0725c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.046962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-956c11a1-8276-4432-ac71-636ab19ade16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c15cd57ca6843868883d4f081e0725c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.060753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 912/1817] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:21 2022] DELETE /identity/v3/users/5c15cd57ca6843868883d4f081e0725c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:22.065148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fa1c0-ab4d-4d3a-92a7-f7c66d016357 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.065431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fa1c0-ab4d-4d3a-92a7-f7c66d016357 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.065702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fa1c0-ab4d-4d3a-92a7-f7c66d016357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.067182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-df7b09d3-f197-43b7-96b9-33d2ea26c8a6 None None] MFA Rules not processed for user `f9168731dea04eb5b423569f846ab0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:22.068420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df7b09d3-f197-43b7-96b9-33d2ea26c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.084542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0c4fa1c0-ab4d-4d3a-92a7-f7c66d016357 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:22.085721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c4fa1c0-ab4d-4d3a-92a7-f7c66d016357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.113621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:22.137686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 906/1818] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:22.148010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 913/1819] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:22.148852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b09d3-f197-43b7-96b9-33d2ea26c8a6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.150249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df7b09d3-f197-43b7-96b9-33d2ea26c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.158063 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fa1c0-ab4d-4d3a-92a7-f7c66d016357 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.159604 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c4fa1c0-ab4d-4d3a-92a7-f7c66d016357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.206487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b09d3-f197-43b7-96b9-33d2ea26c8a6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.210735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.211357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.211631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.211899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.217118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fa1c0-ab4d-4d3a-92a7-f7c66d016357 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.220618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.221425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd5692-5042-4b51-aa65-8e008f086299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.222031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd5692-5042-4b51-aa65-8e008f086299 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.222179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.222337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd5692-5042-4b51-aa65-8e008f086299 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.222611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd5692-5042-4b51-aa65-8e008f086299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.230786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 907/1820] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:22.231838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd5692-5042-4b51-aa65-8e008f086299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.233350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd5692-5042-4b51-aa65-8e008f086299 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.243382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 914/1821] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:22.255625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.257076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.271731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c8ecb-433d-439d-8b00-dff9086cbb31 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.276851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.277469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.277768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.278039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] PATH_INFO: `/v3/users/f9168731dea04eb5b423569f846ab0a6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.279223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9168731dea04eb5b423569f846ab0a6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.281176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.342293 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9168731dea04eb5b423569f846ab0a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.343360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9168731dea04eb5b423569f846ab0a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.357926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9168731dea04eb5b423569f846ab0a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.366440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9168731dea04eb5b423569f846ab0a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.367282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9168731dea04eb5b423569f846ab0a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.381685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 908/1822] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:22 2022] DELETE /identity/v3/users/f9168731dea04eb5b423569f846ab0a6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:22.386637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dfc8e-7ae0-4bf9-a658-d54b8ca58f56 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.386919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dfc8e-7ae0-4bf9-a658-d54b8ca58f56 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.387188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dfc8e-7ae0-4bf9-a658-d54b8ca58f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.406249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-740dfc8e-7ae0-4bf9-a658-d54b8ca58f56 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:22.407503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-740dfc8e-7ae0-4bf9-a658-d54b8ca58f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.491761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 915/1823] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:22.504532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.505996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.579345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec987cf2-00d6-4d2d-acc0-e1db31e725f4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.583295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42128a85-762b-432d-bd0e-6743efbb0d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.584005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42128a85-762b-432d-bd0e-6743efbb0d15 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.584182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42128a85-762b-432d-bd0e-6743efbb0d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.584455 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42128a85-762b-432d-bd0e-6743efbb0d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.589681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dfc8e-7ae0-4bf9-a658-d54b8ca58f56 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.591275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-740dfc8e-7ae0-4bf9-a658-d54b8ca58f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.605159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dfc8e-7ae0-4bf9-a658-d54b8ca58f56 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.608737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42128a85-762b-432d-bd0e-6743efbb0d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.610591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42128a85-762b-432d-bd0e-6743efbb0d15 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.625393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 909/1824] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:22.626401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.627114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.627461 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.627809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] PATH_INFO: `/v3/projects/de61e70413c34586a8fa98b33ea252e0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.629075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de61e70413c34586a8fa98b33ea252e0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.631035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.656380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:22.685614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de61e70413c34586a8fa98b33ea252e0', 'request_id': 'req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.694913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de61e70413c34586a8fa98b33ea252e0', 'request_id': 'req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.701896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de61e70413c34586a8fa98b33ea252e0', 'request_id': 'req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.714424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] The token cache is being invalidate because project de61e70413c34586a8fa98b33ea252e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:22.714709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.715386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8706d999-dfdb-4e7d-9cc8-b6157e8f7c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.716904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 916/1825] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:22 2022] DELETE /identity/v3/projects/de61e70413c34586a8fa98b33ea252e0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:22.722425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb10eb-0df2-4214-9700-b53674875ad2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.722713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb10eb-0df2-4214-9700-b53674875ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.722979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb10eb-0df2-4214-9700-b53674875ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.733079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:22.748435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-01fb10eb-0df2-4214-9700-b53674875ad2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:22.749636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-01fb10eb-0df2-4214-9700-b53674875ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.814707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 910/1826] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:22.822594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb8e8-f869-44f4-855c-ffc4bb84785f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.822879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb8e8-f869-44f4-855c-ffc4bb84785f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.823154 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb8e8-f869-44f4-855c-ffc4bb84785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.843651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-165eb8e8-f869-44f4-855c-ffc4bb84785f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:22.845166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-165eb8e8-f869-44f4-855c-ffc4bb84785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.853120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 917/1827] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:22.857463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb10eb-0df2-4214-9700-b53674875ad2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.867689 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb10eb-0df2-4214-9700-b53674875ad2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.873004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.873610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.873892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.874195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.874874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:22.875179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.876935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.891448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 911/1828] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:22.900879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165eb8e8-f869-44f4-855c-ffc4bb84785f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.905288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.907409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.909714 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165eb8e8-f869-44f4-855c-ffc4bb84785f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.916688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bf24a-fae5-4ade-b11f-ce273aab0391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.918129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf24a-fae5-4ade-b11f-ce273aab0391 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.918779 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf24a-fae5-4ade-b11f-ce273aab0391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.919203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bf24a-fae5-4ade-b11f-ce273aab0391 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.920132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bf24a-fae5-4ade-b11f-ce273aab0391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.921033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09f7c2-99d7-42e5-aa80-bae3721720b2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.922939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bf24a-fae5-4ade-b11f-ce273aab0391 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.931376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.932291 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.932405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.932957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] PATH_INFO: `/v3/projects/8f6ae36207a4410e94904ec198c0b1c2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.934826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6ae36207a4410e94904ec198c0b1c2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.937642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.954297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 918/1829] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:22.958377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bf24a-fae5-4ade-b11f-ce273aab0391 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.990405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6ae36207a4410e94904ec198c0b1c2', 'request_id': 'req-00d335cf-b02e-4d05-b25c-4addf6d22b46'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:22.998110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6ae36207a4410e94904ec198c0b1c2', 'request_id': 'req-00d335cf-b02e-4d05-b25c-4addf6d22b46'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.009051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6ae36207a4410e94904ec198c0b1c2', 'request_id': 'req-00d335cf-b02e-4d05-b25c-4addf6d22b46'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.020757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] The token cache is being invalidate because project 8f6ae36207a4410e94904ec198c0b1c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:23.021119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.021831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.023558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 912/1830] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:22 2022] DELETE /identity/v3/projects/8f6ae36207a4410e94904ec198c0b1c2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:23.037103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bf24a-fae5-4ade-b11f-ce273aab0391 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.110889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a58e9-086c-4c0b-aae3-e76bfe0686bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.111516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a58e9-086c-4c0b-aae3-e76bfe0686bf admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.111798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a58e9-086c-4c0b-aae3-e76bfe0686bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.112071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a58e9-086c-4c0b-aae3-e76bfe0686bf admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.112723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a58e9-086c-4c0b-aae3-e76bfe0686bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.114757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a58e9-086c-4c0b-aae3-e76bfe0686bf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.149879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 919/1831] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:23.150974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.152703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.169667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d335cf-b02e-4d05-b25c-4addf6d22b46 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.175240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909c045-5fc3-4a46-902d-54cdb83faf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.176128 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c045-5fc3-4a46-902d-54cdb83faf48 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.177538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c045-5fc3-4a46-902d-54cdb83faf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.177960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c045-5fc3-4a46-902d-54cdb83faf48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.178961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909c045-5fc3-4a46-902d-54cdb83faf48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.180918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909c045-5fc3-4a46-902d-54cdb83faf48 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.195928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 913/1832] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:23.200477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a58e9-086c-4c0b-aae3-e76bfe0686bf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.211333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a58e9-086c-4c0b-aae3-e76bfe0686bf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.216045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67844df5-6d5f-4a68-8b5d-a9ba441d5735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.216654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67844df5-6d5f-4a68-8b5d-a9ba441d5735 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.216929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67844df5-6d5f-4a68-8b5d-a9ba441d5735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.217197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67844df5-6d5f-4a68-8b5d-a9ba441d5735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.217842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67844df5-6d5f-4a68-8b5d-a9ba441d5735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.220517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67844df5-6d5f-4a68-8b5d-a9ba441d5735 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.231068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 920/1833] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:23.235798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909c045-5fc3-4a46-902d-54cdb83faf48 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.245985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909c045-5fc3-4a46-902d-54cdb83faf48 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.251081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede63e85-53d0-40e6-aeb4-3fb22a035846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.251734 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede63e85-53d0-40e6-aeb4-3fb22a035846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.252014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede63e85-53d0-40e6-aeb4-3fb22a035846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.252289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede63e85-53d0-40e6-aeb4-3fb22a035846 admin admin] PATH_INFO: `/v3/projects/246fabb9f7e445658e8380f74ec5666b/users/b52ebbf571aa482da31fdfc75fae023c/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.267467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede63e85-53d0-40e6-aeb4-3fb22a035846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246fabb9f7e445658e8380f74ec5666b, user_id=b52ebbf571aa482da31fdfc75fae023c, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.270144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede63e85-53d0-40e6-aeb4-3fb22a035846 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.286214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 914/1834] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:23 2022] PUT /identity/v3/projects/246fabb9f7e445658e8380f74ec5666b/users/b52ebbf571aa482da31fdfc75fae023c/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:23.296689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45400b-0324-4f60-903b-349851fee5ad None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.296988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45400b-0324-4f60-903b-349851fee5ad None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.297340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45400b-0324-4f60-903b-349851fee5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.306639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:23.331401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7b45400b-0324-4f60-903b-349851fee5ad None None] MFA Rules not processed for user `b52ebbf571aa482da31fdfc75fae023c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:23.332810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7b45400b-0324-4f60-903b-349851fee5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.399890 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 921/1835] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:23.404473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede63e85-53d0-40e6-aeb4-3fb22a035846 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.453715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:23.479073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede63e85-53d0-40e6-aeb4-3fb22a035846 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.484796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb6bcd-b678-42fc-b62e-b74e9478e8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.485870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb6bcd-b678-42fc-b62e-b74e9478e8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.486151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb6bcd-b678-42fc-b62e-b74e9478e8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.486531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb6bcd-b678-42fc-b62e-b74e9478e8f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.487235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb6bcd-b678-42fc-b62e-b74e9478e8f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.489350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb6bcd-b678-42fc-b62e-b74e9478e8f6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.520259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 915/1836] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:23.524369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45400b-0324-4f60-903b-349851fee5ad None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.537314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45400b-0324-4f60-903b-349851fee5ad None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.616515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5f6eb-0ab2-49a6-a4e9-2a2f7194251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.617260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5f6eb-0ab2-49a6-a4e9-2a2f7194251a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.617542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5f6eb-0ab2-49a6-a4e9-2a2f7194251a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.617922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5f6eb-0ab2-49a6-a4e9-2a2f7194251a admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.618676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5f6eb-0ab2-49a6-a4e9-2a2f7194251a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.620789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5f6eb-0ab2-49a6-a4e9-2a2f7194251a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.663344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 922/1837] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:23.664432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb6bcd-b678-42fc-b62e-b74e9478e8f6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.680334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb6bcd-b678-42fc-b62e-b74e9478e8f6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.687364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fc1f5-ad7a-4cc9-8036-215120d9af22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.688776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fc1f5-ad7a-4cc9-8036-215120d9af22 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.689419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fc1f5-ad7a-4cc9-8036-215120d9af22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.689906 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fc1f5-ad7a-4cc9-8036-215120d9af22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.690855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fc1f5-ad7a-4cc9-8036-215120d9af22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.693100 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fc1f5-ad7a-4cc9-8036-215120d9af22 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.714945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 916/1838] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:23.716993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5f6eb-0ab2-49a6-a4e9-2a2f7194251a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.730521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5f6eb-0ab2-49a6-a4e9-2a2f7194251a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.736692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bede85-0e60-4001-816d-8ba2915f676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.737492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bede85-0e60-4001-816d-8ba2915f676a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.737772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bede85-0e60-4001-816d-8ba2915f676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.738112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bede85-0e60-4001-816d-8ba2915f676a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.738848 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bede85-0e60-4001-816d-8ba2915f676a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.740665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bede85-0e60-4001-816d-8ba2915f676a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.758522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 923/1839] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:23.762020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fc1f5-ad7a-4cc9-8036-215120d9af22 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.775397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fc1f5-ad7a-4cc9-8036-215120d9af22 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.780775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a27456-b866-48c8-a39a-b0d205b1018b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.781744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a27456-b866-48c8-a39a-b0d205b1018b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.782138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a27456-b866-48c8-a39a-b0d205b1018b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.782584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a27456-b866-48c8-a39a-b0d205b1018b admin admin] PATH_INFO: `/v3/projects/4739d61ad2854c939dfa832f191d6c4f/users/49b12be56b0e43219ec22fdf3f55f52f/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.800926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a27456-b866-48c8-a39a-b0d205b1018b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4739d61ad2854c939dfa832f191d6c4f, user_id=49b12be56b0e43219ec22fdf3f55f52f, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.803693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a27456-b866-48c8-a39a-b0d205b1018b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.818604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 917/1840] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:23 2022] PUT /identity/v3/projects/4739d61ad2854c939dfa832f191d6c4f/users/49b12be56b0e43219ec22fdf3f55f52f/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:23.829731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ffb69-ccd3-44e5-b5f6-2a6c2dc85218 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.830009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ffb69-ccd3-44e5-b5f6-2a6c2dc85218 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.830301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ffb69-ccd3-44e5-b5f6-2a6c2dc85218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.860269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3a3ffb69-ccd3-44e5-b5f6-2a6c2dc85218 None None] MFA Rules not processed for user `49b12be56b0e43219ec22fdf3f55f52f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:23.861626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3a3ffb69-ccd3-44e5-b5f6-2a6c2dc85218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.933306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 924/1841] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:23.938138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a27456-b866-48c8-a39a-b0d205b1018b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.013811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a27456-b866-48c8-a39a-b0d205b1018b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.018803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dd783-998f-4624-89b6-8e16e69db798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.019421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd783-998f-4624-89b6-8e16e69db798 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.019701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd783-998f-4624-89b6-8e16e69db798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.019979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd783-998f-4624-89b6-8e16e69db798 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.020617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57dd783-998f-4624-89b6-8e16e69db798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.022554 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57dd783-998f-4624-89b6-8e16e69db798 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.050306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 918/1842] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:24.054487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ffb69-ccd3-44e5-b5f6-2a6c2dc85218 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.061256 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:24.071060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ffb69-ccd3-44e5-b5f6-2a6c2dc85218 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.135802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8210c-7772-4930-8ecd-457cc69b9e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.136494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8210c-7772-4930-8ecd-457cc69b9e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.136782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8210c-7772-4930-8ecd-457cc69b9e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.137071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8210c-7772-4930-8ecd-457cc69b9e14 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.137707 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8210c-7772-4930-8ecd-457cc69b9e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.139678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8210c-7772-4930-8ecd-457cc69b9e14 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.163765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 925/1843] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:24 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:24.168209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dd783-998f-4624-89b6-8e16e69db798 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.179774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dd783-998f-4624-89b6-8e16e69db798 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.185614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c927b05-66fa-4d2a-9ee2-b27b771b7d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.185614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c927b05-66fa-4d2a-9ee2-b27b771b7d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.185836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c927b05-66fa-4d2a-9ee2-b27b771b7d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.186125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c927b05-66fa-4d2a-9ee2-b27b771b7d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.186890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c927b05-66fa-4d2a-9ee2-b27b771b7d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.190128 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c927b05-66fa-4d2a-9ee2-b27b771b7d34 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.202394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 919/1844] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:24.206715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8210c-7772-4930-8ecd-457cc69b9e14 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.217597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8210c-7772-4930-8ecd-457cc69b9e14 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.223541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b420e6-c79b-4f13-882e-bfc18cbb24f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.224348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b420e6-c79b-4f13-882e-bfc18cbb24f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.224628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b420e6-c79b-4f13-882e-bfc18cbb24f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.224994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b420e6-c79b-4f13-882e-bfc18cbb24f4 admin admin] PATH_INFO: `/v3/projects/3516f7a8e90448679fac7a725811bf5a/users/a65036d689be4399a3060a2047a90a69/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.241666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b420e6-c79b-4f13-882e-bfc18cbb24f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3516f7a8e90448679fac7a725811bf5a, user_id=a65036d689be4399a3060a2047a90a69, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.244366 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b420e6-c79b-4f13-882e-bfc18cbb24f4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.260146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 926/1845] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:24 2022] PUT /identity/v3/projects/3516f7a8e90448679fac7a725811bf5a/users/a65036d689be4399a3060a2047a90a69/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:24.264095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c927b05-66fa-4d2a-9ee2-b27b771b7d34 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.275795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c927b05-66fa-4d2a-9ee2-b27b771b7d34 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.352027 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924129ff-e37d-445f-9c2b-78738f0dbec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.352637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924129ff-e37d-445f-9c2b-78738f0dbec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.352914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924129ff-e37d-445f-9c2b-78738f0dbec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.353180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924129ff-e37d-445f-9c2b-78738f0dbec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.353911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924129ff-e37d-445f-9c2b-78738f0dbec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.355547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924129ff-e37d-445f-9c2b-78738f0dbec6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.357525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:24.367969 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 920/1846] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:24.372099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b420e6-c79b-4f13-882e-bfc18cbb24f4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.384589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b420e6-c79b-4f13-882e-bfc18cbb24f4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.389525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.390369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.391233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.391515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] PATH_INFO: `/v3/projects/3516f7a8e90448679fac7a725811bf5a/users/a65036d689be4399a3060a2047a90a69/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.393413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3516f7a8e90448679fac7a725811bf5a, user_id=a65036d689be4399a3060a2047a90a69, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.396054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.407305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 927/1847] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:24 2022] PUT /identity/v3/projects/3516f7a8e90448679fac7a725811bf5a/users/a65036d689be4399a3060a2047a90a69/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:24.417436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92803-cd3f-420e-abe0-d6574cf54a53 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.417928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92803-cd3f-420e-abe0-d6574cf54a53 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.418017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92803-cd3f-420e-abe0-d6574cf54a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.448225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8fe92803-cd3f-420e-abe0-d6574cf54a53 None None] MFA Rules not processed for user `a65036d689be4399a3060a2047a90a69`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:24.449347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8fe92803-cd3f-420e-abe0-d6574cf54a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:24.522438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 921/1848] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:24.534126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.536167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:24.598708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cf71e-005b-473f-b9f2-a3f647c111af admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.602587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01795c88-6678-4f3e-bb8c-a88ef7327fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.603199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795c88-6678-4f3e-bb8c-a88ef7327fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.603482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795c88-6678-4f3e-bb8c-a88ef7327fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.603748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795c88-6678-4f3e-bb8c-a88ef7327fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.671972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01795c88-6678-4f3e-bb8c-a88ef7327fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.673703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01795c88-6678-4f3e-bb8c-a88ef7327fc0 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.683151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 928/1849] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:26.171129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4fd6-89da-44e2-9a40-e263dbabd41f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.171654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4fd6-89da-44e2-9a40-e263dbabd41f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.171899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4fd6-89da-44e2-9a40-e263dbabd41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.193141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ecde4fd6-89da-44e2-9a40-e263dbabd41f None None] MFA Rules not processed for user `1d750b1b27d840b38fd541653a61640b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:26.194419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ecde4fd6-89da-44e2-9a40-e263dbabd41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.217517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01795c88-6678-4f3e-bb8c-a88ef7327fc0 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.226450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01795c88-6678-4f3e-bb8c-a88ef7327fc0 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.230421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.231014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.231286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.231549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.236996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:26.251472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 922/1850] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:26.295674 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.297164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.306868 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 929/1851] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:26.987170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde4fd6-89da-44e2-9a40-e263dbabd41f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.998184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde4fd6-89da-44e2-9a40-e263dbabd41f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:27.003168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c5e8-714b-4ebd-9a95-34e8cb99c90c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:27.003883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c5e8-714b-4ebd-9a95-34e8cb99c90c service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.004228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c5e8-714b-4ebd-9a95-34e8cb99c90c service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.004565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c5e8-714b-4ebd-9a95-34e8cb99c90c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.015343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921c5e8-714b-4ebd-9a95-34e8cb99c90c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:27.016630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921c5e8-714b-4ebd-9a95-34e8cb99c90c service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:27.028066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 923/1852] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:30.405421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:30.406949 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.482131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3514343-7f9d-49c9-9c7f-2b3d2b5f568c service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:30.485981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:30.486701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.486978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.487250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] PATH_INFO: `/v3/users/5318b31b1138460a8ce0a705b0b720a0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.488367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5318b31b1138460a8ce0a705b0b720a0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:30.490349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:30.541089 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5318b31b1138460a8ce0a705b0b720a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.542437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5318b31b1138460a8ce0a705b0b720a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.555708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5318b31b1138460a8ce0a705b0b720a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.562755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5318b31b1138460a8ce0a705b0b720a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.563493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-552e42e3-ef2a-4b4c-ac04-b25d554f2bdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5318b31b1138460a8ce0a705b0b720a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.577312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 930/1853] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:30 2022] DELETE /identity/v3/users/5318b31b1138460a8ce0a705b0b720a0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:30.755601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c5e8-714b-4ebd-9a95-34e8cb99c90c service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:30.766968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c5e8-714b-4ebd-9a95-34e8cb99c90c service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:30.835450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:30.836081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.836362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.836710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] PATH_INFO: `/v3/projects/7c651864ae8c4026920a713b4a7d9a33` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.837953 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c651864ae8c4026920a713b4a7d9a33)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:30.840035 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:30.858331 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:30.889417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c651864ae8c4026920a713b4a7d9a33', 'request_id': 'req-a2669d47-0de6-430c-9d01-322d76075cff'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.895973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c651864ae8c4026920a713b4a7d9a33', 'request_id': 'req-a2669d47-0de6-430c-9d01-322d76075cff'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.904976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c651864ae8c4026920a713b4a7d9a33', 'request_id': 'req-a2669d47-0de6-430c-9d01-322d76075cff'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.920820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] The token cache is being invalidate because project 7c651864ae8c4026920a713b4a7d9a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:30.921313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.922458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2669d47-0de6-430c-9d01-322d76075cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:30.925142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 924/1854] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:30 2022] DELETE /identity/v3/projects/7c651864ae8c4026920a713b4a7d9a33 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:30.932504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129c156-fdbe-4974-b2b4-ea9f6a730fb1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:30.932784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129c156-fdbe-4974-b2b4-ea9f6a730fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:30.933077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129c156-fdbe-4974-b2b4-ea9f6a730fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:30.956189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3129c156-fdbe-4974-b2b4-ea9f6a730fb1 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:30.957391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3129c156-fdbe-4974-b2b4-ea9f6a730fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:30.978693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:31.020646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 931/1855] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:31.028940 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0138-8b9c-4741-9da6-83abae49e18c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.029222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0138-8b9c-4741-9da6-83abae49e18c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.029489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0138-8b9c-4741-9da6-83abae49e18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.050394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-467b0138-8b9c-4741-9da6-83abae49e18c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:31.051593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-467b0138-8b9c-4741-9da6-83abae49e18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.061077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 925/1856] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:31.065367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129c156-fdbe-4974-b2b4-ea9f6a730fb1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.075943 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129c156-fdbe-4974-b2b4-ea9f6a730fb1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.081436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.082028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.082332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.082608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.083239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:31.083538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.085132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.097505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 932/1857] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:31.101850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b0138-8b9c-4741-9da6-83abae49e18c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.111617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b0138-8b9c-4741-9da6-83abae49e18c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.116252 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d414ec-68f1-4f9c-9265-ffab7ca9d85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.116838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d414ec-68f1-4f9c-9265-ffab7ca9d85c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.117113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d414ec-68f1-4f9c-9265-ffab7ca9d85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.117380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d414ec-68f1-4f9c-9265-ffab7ca9d85c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.118004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d414ec-68f1-4f9c-9265-ffab7ca9d85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.119916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d414ec-68f1-4f9c-9265-ffab7ca9d85c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.148194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 926/1858] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:31.152499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.163543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fcd6d-9243-42d1-a72b-126234550318 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.224139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.224769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.225084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.225360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.226001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.228051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.251853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 933/1859] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:31.256317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d414ec-68f1-4f9c-9265-ffab7ca9d85c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.268957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d414ec-68f1-4f9c-9265-ffab7ca9d85c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.273939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3dc71-a7b8-4791-b647-e03e9316b41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.274588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3dc71-a7b8-4791-b647-e03e9316b41e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.274877 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3dc71-a7b8-4791-b647-e03e9316b41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.275126 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3dc71-a7b8-4791-b647-e03e9316b41e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.275765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3dc71-a7b8-4791-b647-e03e9316b41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.277469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3dc71-a7b8-4791-b647-e03e9316b41e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.281275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.283151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.290994 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 927/1860] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:31.292002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3dc71-a7b8-4791-b647-e03e9316b41e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.295367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af441ac-b525-4ec9-b9c7-760087e41877 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.301061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.301834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.302141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.302341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3dc71-a7b8-4791-b647-e03e9316b41e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.302457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] PATH_INFO: `/v3/users/b52ebbf571aa482da31fdfc75fae023c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.303530 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b52ebbf571aa482da31fdfc75fae023c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.306662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.307201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b925aa-8c2e-4d2f-a4ff-78224be290d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.307801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b925aa-8c2e-4d2f-a4ff-78224be290d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.308082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b925aa-8c2e-4d2f-a4ff-78224be290d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.308535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b925aa-8c2e-4d2f-a4ff-78224be290d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.309172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b925aa-8c2e-4d2f-a4ff-78224be290d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.310871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b925aa-8c2e-4d2f-a4ff-78224be290d4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.324985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 928/1861] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:31.326311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b925aa-8c2e-4d2f-a4ff-78224be290d4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.337307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b925aa-8c2e-4d2f-a4ff-78224be290d4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.342730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d837b-8d5c-4c00-8b6d-a48baeca54fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.343682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d837b-8d5c-4c00-8b6d-a48baeca54fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.344163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d837b-8d5c-4c00-8b6d-a48baeca54fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.345006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d837b-8d5c-4c00-8b6d-a48baeca54fa admin admin] PATH_INFO: `/v3/projects/0d2056a08a324aed92eca4be58e888d6/users/08c9e3cb6c164e72b0d55a2724469440/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.358968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b52ebbf571aa482da31fdfc75fae023c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:31.359784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b52ebbf571aa482da31fdfc75fae023c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:31.360318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d837b-8d5c-4c00-8b6d-a48baeca54fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d2056a08a324aed92eca4be58e888d6, user_id=08c9e3cb6c164e72b0d55a2724469440, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.363079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d837b-8d5c-4c00-8b6d-a48baeca54fa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.373230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b52ebbf571aa482da31fdfc75fae023c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:31.377415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 929/1862] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:31 2022] PUT /identity/v3/projects/0d2056a08a324aed92eca4be58e888d6/users/08c9e3cb6c164e72b0d55a2724469440/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:31.380960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b52ebbf571aa482da31fdfc75fae023c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:31.381649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9a278999-900a-4e61-b96f-4767bc876705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b52ebbf571aa482da31fdfc75fae023c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:31.382764 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404e95-aa91-4b4d-9f5e-7a26a25176ca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.383076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404e95-aa91-4b4d-9f5e-7a26a25176ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.383368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404e95-aa91-4b4d-9f5e-7a26a25176ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.396871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 934/1863] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:31 2022] DELETE /identity/v3/users/b52ebbf571aa482da31fdfc75fae023c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:31.401222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205cc4-daf3-4646-b936-df82bcea5e84 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.401507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205cc4-daf3-4646-b936-df82bcea5e84 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.401784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205cc4-daf3-4646-b936-df82bcea5e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.404528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d8404e95-aa91-4b4d-9f5e-7a26a25176ca None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:31.405922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8404e95-aa91-4b4d-9f5e-7a26a25176ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.420430 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-35205cc4-daf3-4646-b936-df82bcea5e84 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:31.421625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-35205cc4-daf3-4646-b936-df82bcea5e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.430666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:31.473725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 930/1864] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:31.480483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8404e95-aa91-4b4d-9f5e-7a26a25176ca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.481917 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8404e95-aa91-4b4d-9f5e-7a26a25176ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.485119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 935/1865] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:31.494073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35205cc4-daf3-4646-b936-df82bcea5e84 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.495449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-35205cc4-daf3-4646-b936-df82bcea5e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:31.519398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:31.545469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8404e95-aa91-4b4d-9f5e-7a26a25176ca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.549357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.549965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.550515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.550793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.555273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35205cc4-daf3-4646-b936-df82bcea5e84 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:31.558877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71899a33-4683-4222-bd71-efaa18e75746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:31.559511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71899a33-4683-4222-bd71-efaa18e75746 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:31.559787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71899a33-4683-4222-bd71-efaa18e75746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:31.560059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71899a33-4683-4222-bd71-efaa18e75746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:31.562000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.563667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.569808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71899a33-4683-4222-bd71-efaa18e75746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:31.571593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71899a33-4683-4222-bd71-efaa18e75746 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:31.573245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 931/1866] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:31.581993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 936/1867] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:31.988505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:31.990723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:32.006526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2cf18-9bd1-4dd4-8025-8a36c4857604 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.019395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.020450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.020950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.021659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] PATH_INFO: `/v3/projects/246fabb9f7e445658e8380f74ec5666b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.023071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=246fabb9f7e445658e8380f74ec5666b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.025460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.088668 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246fabb9f7e445658e8380f74ec5666b', 'request_id': 'req-8a951b42-ec62-474a-a13f-5fdbe87666ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.096755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246fabb9f7e445658e8380f74ec5666b', 'request_id': 'req-8a951b42-ec62-474a-a13f-5fdbe87666ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.107478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246fabb9f7e445658e8380f74ec5666b', 'request_id': 'req-8a951b42-ec62-474a-a13f-5fdbe87666ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.136507 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] The token cache is being invalidate because project 246fabb9f7e445658e8380f74ec5666b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:32.136807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.137486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.139210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 932/1868] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:31 2022] DELETE /identity/v3/projects/246fabb9f7e445658e8380f74ec5666b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:32.143350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71899a33-4683-4222-bd71-efaa18e75746 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:32.153031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71899a33-4683-4222-bd71-efaa18e75746 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:32.211228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:32.211858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:32.212131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:32.212398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] PATH_INFO: `/v3/users/49b12be56b0e43219ec22fdf3f55f52f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:32.213510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b12be56b0e43219ec22fdf3f55f52f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:32.215449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:32.266510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b12be56b0e43219ec22fdf3f55f52f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.267121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b12be56b0e43219ec22fdf3f55f52f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.277187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:32.279592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b12be56b0e43219ec22fdf3f55f52f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.288907 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b12be56b0e43219ec22fdf3f55f52f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.289615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b12be56b0e43219ec22fdf3f55f52f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:32.303902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 937/1869] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:32 2022] DELETE /identity/v3/users/49b12be56b0e43219ec22fdf3f55f52f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:33.051846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.054596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:33.080233 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951b42-ec62-474a-a13f-5fdbe87666ee admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.160540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.161403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.161938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.162376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] PATH_INFO: `/v3/projects/4739d61ad2854c939dfa832f191d6c4f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.163728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4739d61ad2854c939dfa832f191d6c4f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.165941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.224138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4739d61ad2854c939dfa832f191d6c4f', 'request_id': 'req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.234071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4739d61ad2854c939dfa832f191d6c4f', 'request_id': 'req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.241399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:33.250803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4739d61ad2854c939dfa832f191d6c4f', 'request_id': 'req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.266492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] The token cache is being invalidate because project 4739d61ad2854c939dfa832f191d6c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:33.266771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.267433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.269771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 933/1870] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:33 2022] DELETE /identity/v3/projects/4739d61ad2854c939dfa832f191d6c4f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:33.273616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.282194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:33.306879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23869d6-472c-4150-b583-115a8ca7862f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.372641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.373353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.373639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.373906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] PATH_INFO: `/v3/users/a65036d689be4399a3060a2047a90a69` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.375880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a65036d689be4399a3060a2047a90a69)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.377617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.443945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a65036d689be4399a3060a2047a90a69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.445413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a65036d689be4399a3060a2047a90a69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.482229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a65036d689be4399a3060a2047a90a69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.490631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a65036d689be4399a3060a2047a90a69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.491226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dd8f2eb7-a3c4-4c49-abd1-64b1fd17c049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a65036d689be4399a3060a2047a90a69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.513224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 938/1871] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:33 2022] DELETE /identity/v3/users/a65036d689be4399a3060a2047a90a69 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:33.825690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:33.827281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:33.842362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced25ee-3bf8-4679-afd4-b28e5d22b5f6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:33.915026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:33.916320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:33.916651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:33.916921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] PATH_INFO: `/v3/projects/3516f7a8e90448679fac7a725811bf5a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:33.918391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3516f7a8e90448679fac7a725811bf5a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:33.920334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:33.979138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3516f7a8e90448679fac7a725811bf5a', 'request_id': 'req-edbad80a-086e-4081-aaf6-813b49360c31'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.987112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3516f7a8e90448679fac7a725811bf5a', 'request_id': 'req-edbad80a-086e-4081-aaf6-813b49360c31'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:33.997010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3516f7a8e90448679fac7a725811bf5a', 'request_id': 'req-edbad80a-086e-4081-aaf6-813b49360c31'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:34.010060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] The token cache is being invalidate because project 3516f7a8e90448679fac7a725811bf5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:34.010371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:34.011054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edbad80a-086e-4081-aaf6-813b49360c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:34.012668 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 934/1872] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:33 2022] DELETE /identity/v3/projects/3516f7a8e90448679fac7a725811bf5a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:34.019044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3e40-1590-4578-ba41-682ae19ac86b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.019436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3e40-1590-4578-ba41-682ae19ac86b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.019811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3e40-1590-4578-ba41-682ae19ac86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.042357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-169b3e40-1590-4578-ba41-682ae19ac86b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:34.043588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-169b3e40-1590-4578-ba41-682ae19ac86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:34.105734 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:34.117259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 939/1873] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:34.128055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8cb2c-8316-4a45-b95b-33640c9aaaad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.129265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8cb2c-8316-4a45-b95b-33640c9aaaad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.129929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8cb2c-8316-4a45-b95b-33640c9aaaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.157041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f0a8cb2c-8316-4a45-b95b-33640c9aaaad None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:34.158314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f0a8cb2c-8316-4a45-b95b-33640c9aaaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:34.173185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 935/1874] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:34.177808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b3e40-1590-4578-ba41-682ae19ac86b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.192751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b3e40-1590-4578-ba41-682ae19ac86b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.197900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.198579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.198853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.199120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.199772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:34.200074 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.201917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.228130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 940/1875] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:29:34.232491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8cb2c-8316-4a45-b95b-33640c9aaaad None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.244165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8cb2c-8316-4a45-b95b-33640c9aaaad None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.248899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7cb68-c0a7-4ebc-8dd2-0881cc2e9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.249588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7cb68-c0a7-4ebc-8dd2-0881cc2e9c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.250458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7cb68-c0a7-4ebc-8dd2-0881cc2e9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.250747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7cb68-c0a7-4ebc-8dd2-0881cc2e9c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.251399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7cb68-c0a7-4ebc-8dd2-0881cc2e9c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.253342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7cb68-c0a7-4ebc-8dd2-0881cc2e9c9b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.285896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 936/1876] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:34.290814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.310079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ac888-7282-4bbe-b516-6e804477cd8b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.377527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb9153-c64d-4a4e-9105-ca2c975e1733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.378358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb9153-c64d-4a4e-9105-ca2c975e1733 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.378669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb9153-c64d-4a4e-9105-ca2c975e1733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.379002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb9153-c64d-4a4e-9105-ca2c975e1733 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.379712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb9153-c64d-4a4e-9105-ca2c975e1733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.381723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb9153-c64d-4a4e-9105-ca2c975e1733 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.408852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 941/1877] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:34.413352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7cb68-c0a7-4ebc-8dd2-0881cc2e9c9b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.424808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7cb68-c0a7-4ebc-8dd2-0881cc2e9c9b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.430435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb2608-f96c-4651-a066-04b131ba59b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.430435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb2608-f96c-4651-a066-04b131ba59b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.430823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb2608-f96c-4651-a066-04b131ba59b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.431099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb2608-f96c-4651-a066-04b131ba59b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.431798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb2608-f96c-4651-a066-04b131ba59b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.433662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb2608-f96c-4651-a066-04b131ba59b6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.446627 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 937/1878] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:34.449280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb9153-c64d-4a4e-9105-ca2c975e1733 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.461821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb9153-c64d-4a4e-9105-ca2c975e1733 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.466859 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66aaf0-24a0-42e8-afec-886cf75e15a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.467468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66aaf0-24a0-42e8-afec-886cf75e15a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.467735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66aaf0-24a0-42e8-afec-886cf75e15a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.468007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66aaf0-24a0-42e8-afec-886cf75e15a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.468773 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66aaf0-24a0-42e8-afec-886cf75e15a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.470853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66aaf0-24a0-42e8-afec-886cf75e15a5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.485524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 942/1879] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:34.486786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb2608-f96c-4651-a066-04b131ba59b6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.499049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb2608-f96c-4651-a066-04b131ba59b6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.506186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.507124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.507409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.507983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] PATH_INFO: `/v3/projects/8362ccd96d2b47ed852d79b02a042b9d/users/9c081180446a44a698c6ebf8725160b8/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.525016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8362ccd96d2b47ed852d79b02a042b9d, user_id=9c081180446a44a698c6ebf8725160b8, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.528498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.544501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 938/1880] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:34 2022] PUT /identity/v3/projects/8362ccd96d2b47ed852d79b02a042b9d/users/9c081180446a44a698c6ebf8725160b8/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:34.555157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc28cdb-930c-4ad5-b701-462570bb6713 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.555441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc28cdb-930c-4ad5-b701-462570bb6713 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.555712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc28cdb-930c-4ad5-b701-462570bb6713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.602828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4fc28cdb-930c-4ad5-b701-462570bb6713 None None] MFA Rules not processed for user `9c081180446a44a698c6ebf8725160b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:34.604264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4fc28cdb-930c-4ad5-b701-462570bb6713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:34.710532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 943/1881] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:34.718148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:34.722173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:34.742981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:34.821657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b505000-fdc1-4e99-9fc5-d8615ce3533b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:34.825355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4644b-a764-46ae-9be1-1e7dfa4d02a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:34.826170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4644b-a764-46ae-9be1-1e7dfa4d02a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:34.826454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4644b-a764-46ae-9be1-1e7dfa4d02a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:34.826725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4644b-a764-46ae-9be1-1e7dfa4d02a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:34.836186 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4644b-a764-46ae-9be1-1e7dfa4d02a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:34.837661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4644b-a764-46ae-9be1-1e7dfa4d02a4 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:34.847018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 939/1882] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:36.278127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428394a-a513-4794-a282-e00c4561e4bb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:36.278726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428394a-a513-4794-a282-e00c4561e4bb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:36.278726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428394a-a513-4794-a282-e00c4561e4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:36.301050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1428394a-a513-4794-a282-e00c4561e4bb None None] MFA Rules not processed for user `9c081180446a44a698c6ebf8725160b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:36.302509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1428394a-a513-4794-a282-e00c4561e4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:36.311341 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 944/1883] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:37.039903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe022fe-1e91-4996-bdaf-7c91d9b8cbe4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:37.040397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe022fe-1e91-4996-bdaf-7c91d9b8cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:37.040674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe022fe-1e91-4996-bdaf-7c91d9b8cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:37.082868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2fe022fe-1e91-4996-bdaf-7c91d9b8cbe4 None None] MFA Rules not processed for user `08c9e3cb6c164e72b0d55a2724469440`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:37.084359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2fe022fe-1e91-4996-bdaf-7c91d9b8cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:37.226861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 940/1884] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:37.238108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a7e9d-e799-4f84-8366-0dc690759e2f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:37.238594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a7e9d-e799-4f84-8366-0dc690759e2f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:37.239050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a7e9d-e799-4f84-8366-0dc690759e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:37.262601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e24a7e9d-e799-4f84-8366-0dc690759e2f None None] MFA Rules not processed for user `08c9e3cb6c164e72b0d55a2724469440`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:37.263771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e24a7e9d-e799-4f84-8366-0dc690759e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:37.275030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 945/1885] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:37.290795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe022fe-1e91-4996-bdaf-7c91d9b8cbe4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:37.292376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2fe022fe-1e91-4996-bdaf-7c91d9b8cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:37.384222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe022fe-1e91-4996-bdaf-7c91d9b8cbe4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:37.391221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a794748-5b72-4f80-9db1-96934c6ef350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:37.391846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a794748-5b72-4f80-9db1-96934c6ef350 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:37.392122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a794748-5b72-4f80-9db1-96934c6ef350 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:37.392390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a794748-5b72-4f80-9db1-96934c6ef350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:37.401096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a794748-5b72-4f80-9db1-96934c6ef350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:37.402956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a794748-5b72-4f80-9db1-96934c6ef350 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:37.413109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 941/1886] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:38.195520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e127cfe-b33b-47cc-aa03-3129b832a873 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:38.196043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e127cfe-b33b-47cc-aa03-3129b832a873 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:38.196339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e127cfe-b33b-47cc-aa03-3129b832a873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:38.238547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1e127cfe-b33b-47cc-aa03-3129b832a873 None None] MFA Rules not processed for user `08c9e3cb6c164e72b0d55a2724469440`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:38.241631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1e127cfe-b33b-47cc-aa03-3129b832a873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:38.264792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 946/1887] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:29:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:47.408847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:48.219789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:01.970361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6ae90-1f16-4eb4-aa9e-0425a6a95b4e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.970878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6ae90-1f16-4eb4-aa9e-0425a6a95b4e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.970974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6ae90-1f16-4eb4-aa9e-0425a6a95b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.992936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a9c6ae90-1f16-4eb4-aa9e-0425a6a95b4e None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:01.994196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a9c6ae90-1f16-4eb4-aa9e-0425a6a95b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.057024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 942/1888] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:30:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:02.103047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e127cfe-b33b-47cc-aa03-3129b832a873 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.104864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1e127cfe-b33b-47cc-aa03-3129b832a873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.182077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e127cfe-b33b-47cc-aa03-3129b832a873 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.188125 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e043d7-0785-41d0-9de0-0bbed11e7f5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.188732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e043d7-0785-41d0-9de0-0bbed11e7f5b service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.189007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e043d7-0785-41d0-9de0-0bbed11e7f5b service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.189280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e043d7-0785-41d0-9de0-0bbed11e7f5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.198396 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e043d7-0785-41d0-9de0-0bbed11e7f5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.199957 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e043d7-0785-41d0-9de0-0bbed11e7f5b service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.209740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 947/1889] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:30:08.071859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d34b0b-13c6-416a-aaf0-d25830d0599f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.072437 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d34b0b-13c6-416a-aaf0-d25830d0599f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.072437 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d34b0b-13c6-416a-aaf0-d25830d0599f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.100017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-94d34b0b-13c6-416a-aaf0-d25830d0599f None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:08.101219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-94d34b0b-13c6-416a-aaf0-d25830d0599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:08.109951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 943/1890] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:30:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:08.123768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e043d7-0785-41d0-9de0-0bbed11e7f5b service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.139113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e043d7-0785-41d0-9de0-0bbed11e7f5b service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.148204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc0787-3b90-4a44-835e-2948bace8ed9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.149019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc0787-3b90-4a44-835e-2948bace8ed9 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.149451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc0787-3b90-4a44-835e-2948bace8ed9 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.149874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc0787-3b90-4a44-835e-2948bace8ed9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.159085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc0787-3b90-4a44-835e-2948bace8ed9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.160959 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc0787-3b90-4a44-835e-2948bace8ed9 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.177063 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 948/1891] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:30:08.854678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35fefe-b01c-40e5-b03e-9f10418b5162 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.855052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35fefe-b01c-40e5-b03e-9f10418b5162 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.855392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35fefe-b01c-40e5-b03e-9f10418b5162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.881335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-da35fefe-b01c-40e5-b03e-9f10418b5162 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:08.882495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-da35fefe-b01c-40e5-b03e-9f10418b5162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:08.890148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 944/1892] 10.222.0.25 () {58 vars in 967 bytes} [Wed Nov 2 07:30:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:08.905953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc0787-3b90-4a44-835e-2948bace8ed9 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.919823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc0787-3b90-4a44-835e-2948bace8ed9 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.926571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faf07f-aa8b-4d32-a8ac-24077c0cd912 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.927401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faf07f-aa8b-4d32-a8ac-24077c0cd912 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.927871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faf07f-aa8b-4d32-a8ac-24077c0cd912 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.928448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faf07f-aa8b-4d32-a8ac-24077c0cd912 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.938355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faf07f-aa8b-4d32-a8ac-24077c0cd912 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.940297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faf07f-aa8b-4d32-a8ac-24077c0cd912 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.949852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 949/1893] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:30:09.210889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8d513-f6c6-4b27-b3fb-ce13fa911075 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.211532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8d513-f6c6-4b27-b3fb-ce13fa911075 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.211532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8d513-f6c6-4b27-b3fb-ce13fa911075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.233594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-45a8d513-f6c6-4b27-b3fb-ce13fa911075 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:09.235051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-45a8d513-f6c6-4b27-b3fb-ce13fa911075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:09.242762 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 945/1894] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:30:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:09.257933 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faf07f-aa8b-4d32-a8ac-24077c0cd912 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:09.270494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faf07f-aa8b-4d32-a8ac-24077c0cd912 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:09.275693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee70a39-e19c-45f8-9baa-db6921207ec6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:09.276344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee70a39-e19c-45f8-9baa-db6921207ec6 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.276601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee70a39-e19c-45f8-9baa-db6921207ec6 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.277060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee70a39-e19c-45f8-9baa-db6921207ec6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.285942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee70a39-e19c-45f8-9baa-db6921207ec6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:09.287898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee70a39-e19c-45f8-9baa-db6921207ec6 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:09.297538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 950/1895] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:30:09.708421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a042ec-931b-4ba8-a71f-7c58f49c5783 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.708926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a042ec-931b-4ba8-a71f-7c58f49c5783 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.709201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a042ec-931b-4ba8-a71f-7c58f49c5783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.734500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-06a042ec-931b-4ba8-a71f-7c58f49c5783 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:09.735708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-06a042ec-931b-4ba8-a71f-7c58f49c5783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:09.751852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 946/1896] 10.222.0.25 () {58 vars in 967 bytes} [Wed Nov 2 07:30:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:09.761342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee70a39-e19c-45f8-9baa-db6921207ec6 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:09.772745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee70a39-e19c-45f8-9baa-db6921207ec6 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:09.778551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0742ed-a447-4e3e-832d-3ab97424eae1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:09.779166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0742ed-a447-4e3e-832d-3ab97424eae1 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:09.779467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0742ed-a447-4e3e-832d-3ab97424eae1 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:09.779741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0742ed-a447-4e3e-832d-3ab97424eae1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:09.794300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0742ed-a447-4e3e-832d-3ab97424eae1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:09.796443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0742ed-a447-4e3e-832d-3ab97424eae1 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:09.805521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 951/1897] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:30:12.083359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a042ec-931b-4ba8-a71f-7c58f49c5783 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:12.159631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a042ec-931b-4ba8-a71f-7c58f49c5783 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:12.164659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:12.165362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:12.165661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:12.165931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] PATH_INFO: `/v3/users/9c081180446a44a698c6ebf8725160b8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:12.167321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c081180446a44a698c6ebf8725160b8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:12.169354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:12.225288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c081180446a44a698c6ebf8725160b8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.225945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c081180446a44a698c6ebf8725160b8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.244182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c081180446a44a698c6ebf8725160b8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.251581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c081180446a44a698c6ebf8725160b8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.252285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c081180446a44a698c6ebf8725160b8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.263138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:12.270710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 947/1898] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:12 2022] DELETE /identity/v3/users/9c081180446a44a698c6ebf8725160b8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:12.275732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56817a76-fa60-4b1c-8796-09df85addba8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:12.276584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56817a76-fa60-4b1c-8796-09df85addba8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:12.276857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56817a76-fa60-4b1c-8796-09df85addba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:12.305186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-56817a76-fa60-4b1c-8796-09df85addba8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:12.307096 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-56817a76-fa60-4b1c-8796-09df85addba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:12.383075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 952/1899] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:12.393181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:12.395013 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:12.467888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0123215-595c-461d-990b-7d9c0d06fc9c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:12.474899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2c71f-c181-4954-9ba5-5cb89438f4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:12.475924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c71f-c181-4954-9ba5-5cb89438f4bc service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:12.476323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c71f-c181-4954-9ba5-5cb89438f4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:12.476809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c71f-c181-4954-9ba5-5cb89438f4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:12.491149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2c71f-c181-4954-9ba5-5cb89438f4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:12.493169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2c71f-c181-4954-9ba5-5cb89438f4bc service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:12.505479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 948/1900] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:12.715892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56817a76-fa60-4b1c-8796-09df85addba8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:12.717378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-56817a76-fa60-4b1c-8796-09df85addba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:12.764065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56817a76-fa60-4b1c-8796-09df85addba8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:12.772787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:12.773447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:12.773727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:12.773997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] PATH_INFO: `/v3/projects/8362ccd96d2b47ed852d79b02a042b9d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:12.775297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8362ccd96d2b47ed852d79b02a042b9d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:12.777438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:12.823910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:12.841224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8362ccd96d2b47ed852d79b02a042b9d', 'request_id': 'req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.852008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8362ccd96d2b47ed852d79b02a042b9d', 'request_id': 'req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.859910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8362ccd96d2b47ed852d79b02a042b9d', 'request_id': 'req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.883301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] The token cache is being invalidate because project 8362ccd96d2b47ed852d79b02a042b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:12.884099 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.886733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ed71d430-322a-4cba-a3b7-ab5bc321c9c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:12.889900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 953/1901] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:12 2022] DELETE /identity/v3/projects/8362ccd96d2b47ed852d79b02a042b9d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:12.895222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faef1-c966-4db1-94a1-7dd3812c24ea None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:12.895527 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faef1-c966-4db1-94a1-7dd3812c24ea None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:12.895801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faef1-c966-4db1-94a1-7dd3812c24ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:12.923067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-446faef1-c966-4db1-94a1-7dd3812c24ea None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:12.924420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-446faef1-c966-4db1-94a1-7dd3812c24ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:13.027408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 949/1902] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:13.037275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f66e8f-32ec-43f5-9bb1-8a335a322442 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.037711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f66e8f-32ec-43f5-9bb1-8a335a322442 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.038051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f66e8f-32ec-43f5-9bb1-8a335a322442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.068872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-92f66e8f-32ec-43f5-9bb1-8a335a322442 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:13.070626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-92f66e8f-32ec-43f5-9bb1-8a335a322442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:13.084670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 954/1903] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:13.089797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446faef1-c966-4db1-94a1-7dd3812c24ea None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.108447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446faef1-c966-4db1-94a1-7dd3812c24ea None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.115469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.116040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.116314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.116587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.117245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:13.117546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.119717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.140077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 950/1904] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:13.145532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f66e8f-32ec-43f5-9bb1-8a335a322442 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.162198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f66e8f-32ec-43f5-9bb1-8a335a322442 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.169812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b63684-d226-4f32-b2f2-a43a783af1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.169812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b63684-d226-4f32-b2f2-a43a783af1b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.169812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b63684-d226-4f32-b2f2-a43a783af1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.169812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b63684-d226-4f32-b2f2-a43a783af1b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.170556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b63684-d226-4f32-b2f2-a43a783af1b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.172786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b63684-d226-4f32-b2f2-a43a783af1b3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.210583 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 955/1905] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:13 2022] POST /identity/v3/projects => generated 399 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:13.214987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.229878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a031f-eff1-481d-b81b-cdb1a254c369 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.312718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73bdc-73ee-469d-821d-2691ccd21e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.313463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73bdc-73ee-469d-821d-2691ccd21e90 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.313805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73bdc-73ee-469d-821d-2691ccd21e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.314172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73bdc-73ee-469d-821d-2691ccd21e90 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.314879 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf73bdc-73ee-469d-821d-2691ccd21e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.317234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf73bdc-73ee-469d-821d-2691ccd21e90 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.346146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 951/1906] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:13 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:13.350275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b63684-d226-4f32-b2f2-a43a783af1b3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.361656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b63684-d226-4f32-b2f2-a43a783af1b3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.367148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c383ffc-1d05-4afb-8089-2f2c33f4fbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.367759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c383ffc-1d05-4afb-8089-2f2c33f4fbbc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.368038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c383ffc-1d05-4afb-8089-2f2c33f4fbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.368305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c383ffc-1d05-4afb-8089-2f2c33f4fbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.368944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c383ffc-1d05-4afb-8089-2f2c33f4fbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.370610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c383ffc-1d05-4afb-8089-2f2c33f4fbbc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.383060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 956/1907] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:13.387127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73bdc-73ee-469d-821d-2691ccd21e90 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.402517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73bdc-73ee-469d-821d-2691ccd21e90 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.407954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a7257-61fd-4fa6-a297-dbd3a4ae6c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.408590 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7257-61fd-4fa6-a297-dbd3a4ae6c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.408870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7257-61fd-4fa6-a297-dbd3a4ae6c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.409145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7257-61fd-4fa6-a297-dbd3a4ae6c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.409796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a7257-61fd-4fa6-a297-dbd3a4ae6c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.411466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a7257-61fd-4fa6-a297-dbd3a4ae6c19 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.425986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 952/1908] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:13.431531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c383ffc-1d05-4afb-8089-2f2c33f4fbbc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.443751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c383ffc-1d05-4afb-8089-2f2c33f4fbbc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.449494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.450569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.450942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.451522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] PATH_INFO: `/v3/projects/cf54accdb8d0437789609d5e92cb8c99/users/43734012a2b4413fa13ec90b32a24c8b/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.481337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf54accdb8d0437789609d5e92cb8c99, user_id=43734012a2b4413fa13ec90b32a24c8b, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.488374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.507795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 957/1909] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:13 2022] PUT /identity/v3/projects/cf54accdb8d0437789609d5e92cb8c99/users/43734012a2b4413fa13ec90b32a24c8b/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:13.523391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76e968-e2a7-458b-9aa4-830464d2e95f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.523724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76e968-e2a7-458b-9aa4-830464d2e95f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.524177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76e968-e2a7-458b-9aa4-830464d2e95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.538382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:13.561126 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7a76e968-e2a7-458b-9aa4-830464d2e95f None None] MFA Rules not processed for user `43734012a2b4413fa13ec90b32a24c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:13.562857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7a76e968-e2a7-458b-9aa4-830464d2e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:13.645845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 953/1910] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:30:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:13.664485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.666406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:13.731630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c8ab7-cf69-49c5-a5ba-673abed97798 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.735687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.736368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.736702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.737024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.746315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.747921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.759862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 958/1911] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:13 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:22.204439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf58e-839d-49ac-a7c3-1654a6e31aa7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.205000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf58e-839d-49ac-a7c3-1654a6e31aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.205000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf58e-839d-49ac-a7c3-1654a6e31aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.226152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4abcf58e-839d-49ac-a7c3-1654a6e31aa7 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:22.228586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4abcf58e-839d-49ac-a7c3-1654a6e31aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.288615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 954/1912] 10.222.0.25 () {58 vars in 967 bytes} [Wed Nov 2 07:30:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:22.304203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.305631 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.378808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:22.388614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d06eb0-c6fb-44e9-859a-136ce44e3d13 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.395980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae8335-2643-4812-a68c-f276a99f505b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.396563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae8335-2643-4812-a68c-f276a99f505b service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.396876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae8335-2643-4812-a68c-f276a99f505b service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.397144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae8335-2643-4812-a68c-f276a99f505b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.407006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae8335-2643-4812-a68c-f276a99f505b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.408598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae8335-2643-4812-a68c-f276a99f505b service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.417945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 959/1913] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:30:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:30:24.268852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcf58e-839d-49ac-a7c3-1654a6e31aa7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:24.335912 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcf58e-839d-49ac-a7c3-1654a6e31aa7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:24.340421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:24.341052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:24.341332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:24.341606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] PATH_INFO: `/v3/users/43734012a2b4413fa13ec90b32a24c8b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:24.342885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=43734012a2b4413fa13ec90b32a24c8b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:24.345229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:24.392786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:24.395852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43734012a2b4413fa13ec90b32a24c8b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.396682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43734012a2b4413fa13ec90b32a24c8b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.411897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43734012a2b4413fa13ec90b32a24c8b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.418683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43734012a2b4413fa13ec90b32a24c8b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.419565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43734012a2b4413fa13ec90b32a24c8b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.438158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 955/1914] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:24 2022] DELETE /identity/v3/users/43734012a2b4413fa13ec90b32a24c8b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:24.444613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5576-34ab-4a07-a40a-6b6507fdbf53 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:24.445148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5576-34ab-4a07-a40a-6b6507fdbf53 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:24.445372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5576-34ab-4a07-a40a-6b6507fdbf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:24.462331 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:24.464301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:24.467835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8aef5576-34ab-4a07-a40a-6b6507fdbf53 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:24.469000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8aef5576-34ab-4a07-a40a-6b6507fdbf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:24.533289 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33633a4b-83fe-4dc0-b1ea-b98789e071fe admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:24.533991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 960/1915] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:24.538004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:24.538688 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:24.538965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:24.539235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] PATH_INFO: `/v3/users/1d750b1b27d840b38fd541653a61640b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:24.540431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d750b1b27d840b38fd541653a61640b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:24.542405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:24.544261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef5576-34ab-4a07-a40a-6b6507fdbf53 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:24.545817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8aef5576-34ab-4a07-a40a-6b6507fdbf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:24.592725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d750b1b27d840b38fd541653a61640b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.593394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d750b1b27d840b38fd541653a61640b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.605753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef5576-34ab-4a07-a40a-6b6507fdbf53 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:24.606563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d750b1b27d840b38fd541653a61640b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.610085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:24.610727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:24.611005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:24.611275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:24.612999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:24.615139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d750b1b27d840b38fd541653a61640b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.615828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d750b1b27d840b38fd541653a61640b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:24.625344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:24.626805 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:24.630052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 956/1916] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:24 2022] DELETE /identity/v3/users/1d750b1b27d840b38fd541653a61640b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:24.638043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 961/1917] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:24.929779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:24.931445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:24.984506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:25.007427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408e45d-576c-4c81-b835-3c9654d1634b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.012294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.012934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.013215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.013488 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] PATH_INFO: `/v3/projects/cf54accdb8d0437789609d5e92cb8c99` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.014840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf54accdb8d0437789609d5e92cb8c99)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.016670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.073037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.073559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf54accdb8d0437789609d5e92cb8c99', 'request_id': 'req-1a047b12-ee7e-463d-b830-c6f7f33b02f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.074790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:25.089754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb66a-d2c4-4e59-bfef-71fc3aece5ff service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.104522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf54accdb8d0437789609d5e92cb8c99', 'request_id': 'req-1a047b12-ee7e-463d-b830-c6f7f33b02f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.120409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf54accdb8d0437789609d5e92cb8c99', 'request_id': 'req-1a047b12-ee7e-463d-b830-c6f7f33b02f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.133440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] The token cache is being invalidate because project cf54accdb8d0437789609d5e92cb8c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:25.133821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.135372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a047b12-ee7e-463d-b830-c6f7f33b02f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.136842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 957/1918] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:24 2022] DELETE /identity/v3/projects/cf54accdb8d0437789609d5e92cb8c99 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:25.157312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.158241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.158643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.159173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] PATH_INFO: `/v3/projects/1d3fbb20c86e4f9cb73273144f1734d9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.160552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3fbb20c86e4f9cb73273144f1734d9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.162673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.210569 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:25.225729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3fbb20c86e4f9cb73273144f1734d9', 'request_id': 'req-bd02b77e-a94c-4079-bf2f-6abfa48120ba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.242715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3fbb20c86e4f9cb73273144f1734d9', 'request_id': 'req-bd02b77e-a94c-4079-bf2f-6abfa48120ba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.252430 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3fbb20c86e4f9cb73273144f1734d9', 'request_id': 'req-bd02b77e-a94c-4079-bf2f-6abfa48120ba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.264354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] The token cache is being invalidate because project 1d3fbb20c86e4f9cb73273144f1734d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:25.264354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.265475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bd02b77e-a94c-4079-bf2f-6abfa48120ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:25.267092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 962/1919] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:25 2022] DELETE /identity/v3/projects/1d3fbb20c86e4f9cb73273144f1734d9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:25.272202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862731-4939-47da-86b6-824cd5c7cb4c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.272703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862731-4939-47da-86b6-824cd5c7cb4c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.272823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862731-4939-47da-86b6-824cd5c7cb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.294672 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7a862731-4939-47da-86b6-824cd5c7cb4c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:25.296111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7a862731-4939-47da-86b6-824cd5c7cb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:25.363178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 958/1920] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:25.372068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0389d-df6c-4831-8274-9a7ffe6d09c6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.372357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0389d-df6c-4831-8274-9a7ffe6d09c6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.372630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0389d-df6c-4831-8274-9a7ffe6d09c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.398882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eec0389d-df6c-4831-8274-9a7ffe6d09c6 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:25.400811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eec0389d-df6c-4831-8274-9a7ffe6d09c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:25.411328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 963/1921] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:25.415822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a862731-4939-47da-86b6-824cd5c7cb4c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.425699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a862731-4939-47da-86b6-824cd5c7cb4c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.431886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.432519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.432752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.433059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.433692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:25.433998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.435667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.447644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 959/1922] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:30:25.451968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0389d-df6c-4831-8274-9a7ffe6d09c6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.461900 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0389d-df6c-4831-8274-9a7ffe6d09c6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.467038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8ee75-ce65-44b1-893d-bbb5d80079e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.467842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8ee75-ce65-44b1-893d-bbb5d80079e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.468105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8ee75-ce65-44b1-893d-bbb5d80079e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.468404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8ee75-ce65-44b1-893d-bbb5d80079e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.469037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8ee75-ce65-44b1-893d-bbb5d80079e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.471273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8ee75-ce65-44b1-893d-bbb5d80079e4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.503368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 964/1923] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:25 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:25.507686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.518081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7e87c-214f-4a3e-8da3-798d61b302a9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.569518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:25.577101 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78f59b-767e-4bd6-a0db-c6ce6937c118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.577705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78f59b-767e-4bd6-a0db-c6ce6937c118 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.578169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78f59b-767e-4bd6-a0db-c6ce6937c118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.578444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78f59b-767e-4bd6-a0db-c6ce6937c118 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.579086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78f59b-767e-4bd6-a0db-c6ce6937c118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.581391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78f59b-767e-4bd6-a0db-c6ce6937c118 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.606006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 960/1924] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:25 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:25.610405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8ee75-ce65-44b1-893d-bbb5d80079e4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.621550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8ee75-ce65-44b1-893d-bbb5d80079e4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.627187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0a76-8522-4ebb-ba1d-b03b20539740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.627788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0a76-8522-4ebb-ba1d-b03b20539740 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.628061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0a76-8522-4ebb-ba1d-b03b20539740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.628329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0a76-8522-4ebb-ba1d-b03b20539740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.628964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0a76-8522-4ebb-ba1d-b03b20539740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.630611 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0a76-8522-4ebb-ba1d-b03b20539740 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.641699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 965/1925] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:25.647517 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78f59b-767e-4bd6-a0db-c6ce6937c118 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.658428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78f59b-767e-4bd6-a0db-c6ce6937c118 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.662972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6dc88-7538-4711-91a8-5f08d01fd7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.663567 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6dc88-7538-4711-91a8-5f08d01fd7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.663843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6dc88-7538-4711-91a8-5f08d01fd7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.664107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6dc88-7538-4711-91a8-5f08d01fd7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.664743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6dc88-7538-4711-91a8-5f08d01fd7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.666408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6dc88-7538-4711-91a8-5f08d01fd7e5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.676846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 961/1926] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:25.681002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0a76-8522-4ebb-ba1d-b03b20539740 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.691085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0a76-8522-4ebb-ba1d-b03b20539740 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.696065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.696704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.697001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.697270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] PATH_INFO: `/v3/projects/abb0929fa957409ea7c236ea80aac411/users/e2ae998756d042d2b4c38f3b56b49a39/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.713946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb0929fa957409ea7c236ea80aac411, user_id=e2ae998756d042d2b4c38f3b56b49a39, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.716606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.731532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 966/1927] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:25 2022] PUT /identity/v3/projects/abb0929fa957409ea7c236ea80aac411/users/e2ae998756d042d2b4c38f3b56b49a39/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:25.742550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033cafa-0896-453f-94cf-75a224032e17 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.742868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033cafa-0896-453f-94cf-75a224032e17 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.743211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033cafa-0896-453f-94cf-75a224032e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.773916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e033cafa-0896-453f-94cf-75a224032e17 None None] MFA Rules not processed for user `e2ae998756d042d2b4c38f3b56b49a39`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:25.775199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e033cafa-0896-453f-94cf-75a224032e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:25.845327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 962/1928] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:30:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:25.857839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:25.860388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:25.923040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f005064-c16e-429f-be70-4defbd62f1db admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:25.927221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fd09151e284f639c93699b0d26db05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:25.927836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:25.928139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:25.928411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:25.937871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:25.939407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:25.948438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 967/1929] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:35.841262 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:35.944246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:02.396293 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfa669-a2fa-4fdf-ab9f-67af5c35ca22 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.396869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfa669-a2fa-4fdf-ab9f-67af5c35ca22 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.396963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfa669-a2fa-4fdf-ab9f-67af5c35ca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.418928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2adfa669-a2fa-4fdf-ab9f-67af5c35ca22 None None] MFA Rules not processed for user `0e8717d368d44102bea0eb82fbee8a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:02.420147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2adfa669-a2fa-4fdf-ab9f-67af5c35ca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:02.486162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 963/1930] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:03.981147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.986289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.064227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80a56c-29eb-4e19-b1b5-b89c1d026a74 service glance] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.069770 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.070567 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.070844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.071117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.072851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.085388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.087004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.097495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 968/1931] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:12.022131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9dd0-fa40-420e-9af4-a2a30b9964f8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:12.022636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9dd0-fa40-420e-9af4-a2a30b9964f8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:12.022742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9dd0-fa40-420e-9af4-a2a30b9964f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:12.059502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f5aa9dd0-fa40-420e-9af4-a2a30b9964f8 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:12.064054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5aa9dd0-fa40-420e-9af4-a2a30b9964f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:12.135911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 964/1932] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:31:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:12.159753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:12.161829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:12.236906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd38a63-a79b-4342-8ce9-c1d29c7714de service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:12.242404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266f032-489b-4f5b-bbf5-200a51f191be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:12.243018 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266f032-489b-4f5b-bbf5-200a51f191be service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:12.243301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266f032-489b-4f5b-bbf5-200a51f191be service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:12.243577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266f032-489b-4f5b-bbf5-200a51f191be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:12.252899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266f032-489b-4f5b-bbf5-200a51f191be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:12.254581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266f032-489b-4f5b-bbf5-200a51f191be service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:12.263327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 969/1933] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:31:18.059797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9dd0-fa40-420e-9af4-a2a30b9964f8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:18.153149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9dd0-fa40-420e-9af4-a2a30b9964f8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:18.157856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:18.158560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.158837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.159138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] PATH_INFO: `/v3/users/e2ae998756d042d2b4c38f3b56b49a39` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.160287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ae998756d042d2b4c38f3b56b49a39)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:18.162262 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:18.215794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ae998756d042d2b4c38f3b56b49a39'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:18.216566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ae998756d042d2b4c38f3b56b49a39'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:18.222752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:18.235324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ae998756d042d2b4c38f3b56b49a39'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:18.244368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ae998756d042d2b4c38f3b56b49a39'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:18.245410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ae998756d042d2b4c38f3b56b49a39'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:18.261967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 965/1934] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:18 2022] DELETE /identity/v3/users/e2ae998756d042d2b4c38f3b56b49a39 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:18.266782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98228-5bee-4de6-a06a-82235230f395 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.267355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98228-5bee-4de6-a06a-82235230f395 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.267355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc98228-5bee-4de6-a06a-82235230f395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.295375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2bc98228-5bee-4de6-a06a-82235230f395 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:18.296574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2bc98228-5bee-4de6-a06a-82235230f395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:18.408203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 970/1935] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:18.420419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:18.421995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:18.491514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22921b14-84c9-4afb-b2c2-09c1b5445c60 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:18.495148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24a762-c928-4f8c-a9d7-58274bf61c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:18.495763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24a762-c928-4f8c-a9d7-58274bf61c6b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:18.496137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24a762-c928-4f8c-a9d7-58274bf61c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:18.496296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24a762-c928-4f8c-a9d7-58274bf61c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:18.505857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24a762-c928-4f8c-a9d7-58274bf61c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:18.507334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24a762-c928-4f8c-a9d7-58274bf61c6b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:18.520209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 966/1936] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:19.082509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc98228-5bee-4de6-a06a-82235230f395 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:19.084037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2bc98228-5bee-4de6-a06a-82235230f395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:19.098208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc98228-5bee-4de6-a06a-82235230f395 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:19.103590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:19.104431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.104800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.105178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] PATH_INFO: `/v3/projects/abb0929fa957409ea7c236ea80aac411` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.106923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb0929fa957409ea7c236ea80aac411)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:19.109592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:19.142329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:19.185529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb0929fa957409ea7c236ea80aac411', 'request_id': 'req-525fd47d-b294-4d82-93b8-24961b7a7e88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:19.194465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb0929fa957409ea7c236ea80aac411', 'request_id': 'req-525fd47d-b294-4d82-93b8-24961b7a7e88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:19.201979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb0929fa957409ea7c236ea80aac411', 'request_id': 'req-525fd47d-b294-4d82-93b8-24961b7a7e88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:19.213887 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] The token cache is being invalidate because project abb0929fa957409ea7c236ea80aac411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:19.214472 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:19.215122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-525fd47d-b294-4d82-93b8-24961b7a7e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:19.217551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 971/1937] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:19 2022] DELETE /identity/v3/projects/abb0929fa957409ea7c236ea80aac411 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:19.224146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de0035-7f64-4824-a583-32ae9ef618d5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.224693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de0035-7f64-4824-a583-32ae9ef618d5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.224886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de0035-7f64-4824-a583-32ae9ef618d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.247197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-28de0035-7f64-4824-a583-32ae9ef618d5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:19.248587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-28de0035-7f64-4824-a583-32ae9ef618d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:19.318131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 967/1938] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:19.322508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf858-8af1-4d60-aa3d-d9c2824b8961 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.323020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf858-8af1-4d60-aa3d-d9c2824b8961 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.323103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf858-8af1-4d60-aa3d-d9c2824b8961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.345265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8d7cf858-8af1-4d60-aa3d-d9c2824b8961 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:19.346528 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8d7cf858-8af1-4d60-aa3d-d9c2824b8961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:19.355643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 972/1939] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:19.361123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de0035-7f64-4824-a583-32ae9ef618d5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:19.372350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de0035-7f64-4824-a583-32ae9ef618d5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:19.380812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:19.381973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.382300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.382577 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.383226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:31:19.383524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:19.385502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:19.399843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 968/1940] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:31:19.403909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf858-8af1-4d60-aa3d-d9c2824b8961 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:19.415637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf858-8af1-4d60-aa3d-d9c2824b8961 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:19.421147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17241714-2877-4809-9d82-9894082e72e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:19.421789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17241714-2877-4809-9d82-9894082e72e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.422045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17241714-2877-4809-9d82-9894082e72e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.422352 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17241714-2877-4809-9d82-9894082e72e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.422997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17241714-2877-4809-9d82-9894082e72e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:19.424907 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17241714-2877-4809-9d82-9894082e72e0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:19.461697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 973/1941] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:19 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:31:19.465811 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:19.478743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c59249-5e29-4ba0-bb25-d12a15233061 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:19.541110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd61fe7-f58a-45a1-b12f-7c9226912802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:19.542039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd61fe7-f58a-45a1-b12f-7c9226912802 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.542534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd61fe7-f58a-45a1-b12f-7c9226912802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.542810 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd61fe7-f58a-45a1-b12f-7c9226912802 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.543456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd61fe7-f58a-45a1-b12f-7c9226912802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:19.546178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd61fe7-f58a-45a1-b12f-7c9226912802 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:19.554603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:19.574039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 969/1942] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:19 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:19.578940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17241714-2877-4809-9d82-9894082e72e0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:19.590432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17241714-2877-4809-9d82-9894082e72e0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:19.597614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d0f56-fb79-48ca-af39-0fa36f33c8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:19.598546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d0f56-fb79-48ca-af39-0fa36f33c8ad admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.598837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d0f56-fb79-48ca-af39-0fa36f33c8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.599115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d0f56-fb79-48ca-af39-0fa36f33c8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.600318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d0f56-fb79-48ca-af39-0fa36f33c8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:19.602653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d0f56-fb79-48ca-af39-0fa36f33c8ad admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:19.614913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 974/1943] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:31:19.618715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd61fe7-f58a-45a1-b12f-7c9226912802 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:19.633592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd61fe7-f58a-45a1-b12f-7c9226912802 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:19.639606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db9b97-b8ae-47a7-9c36-0dd62cf7f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:19.640246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db9b97-b8ae-47a7-9c36-0dd62cf7f1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.640634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db9b97-b8ae-47a7-9c36-0dd62cf7f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.641104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db9b97-b8ae-47a7-9c36-0dd62cf7f1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.641958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db9b97-b8ae-47a7-9c36-0dd62cf7f1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:19.644323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db9b97-b8ae-47a7-9c36-0dd62cf7f1c5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:19.657212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 970/1944] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:19.661924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d0f56-fb79-48ca-af39-0fa36f33c8ad admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:19.688909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d0f56-fb79-48ca-af39-0fa36f33c8ad admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:19.703118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:19.704068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.705293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.707555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] PATH_INFO: `/v3/projects/c513abe395e34a08ba8fa7d59a5aae74/users/563c6cbc71ee4cabbbb02047766c9af8/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.743234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c513abe395e34a08ba8fa7d59a5aae74, user_id=563c6cbc71ee4cabbbb02047766c9af8, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:19.746154 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:19.773919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 975/1945] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:19 2022] PUT /identity/v3/projects/c513abe395e34a08ba8fa7d59a5aae74/users/563c6cbc71ee4cabbbb02047766c9af8/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:19.779958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c7cdf-4ff7-4317-87da-fb27c8acdf45 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:19.780257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c7cdf-4ff7-4317-87da-fb27c8acdf45 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:19.780533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c7cdf-4ff7-4317-87da-fb27c8acdf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:19.834197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e43c7cdf-4ff7-4317-87da-fb27c8acdf45 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:19.835481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e43c7cdf-4ff7-4317-87da-fb27c8acdf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:19.919039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 971/1946] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:19.929313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:19.931481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:20.023261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f7be7-2bc8-4577-95a8-1b27f566b6a9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:20.028739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cef78-91d7-4ae3-a0e7-21b783d1c1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:20.029903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cef78-91d7-4ae3-a0e7-21b783d1c1be service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:20.030220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cef78-91d7-4ae3-a0e7-21b783d1c1be service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:20.030492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cef78-91d7-4ae3-a0e7-21b783d1c1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:20.041501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cef78-91d7-4ae3-a0e7-21b783d1c1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:20.043057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cef78-91d7-4ae3-a0e7-21b783d1c1be service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:20.069943 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 976/1947] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:23.906316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21bdc-21fd-434d-9ac3-0ff30058819b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:23.906921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21bdc-21fd-434d-9ac3-0ff30058819b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:23.906921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a21bdc-21fd-434d-9ac3-0ff30058819b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:23.928429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-90a21bdc-21fd-434d-9ac3-0ff30058819b None None] MFA Rules not processed for user `08c9e3cb6c164e72b0d55a2724469440`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:23.929634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-90a21bdc-21fd-434d-9ac3-0ff30058819b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:23.993140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 972/1948] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:27.008513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa5c09-c50f-4766-ac17-1ceaca678ac7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.009305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa5c09-c50f-4766-ac17-1ceaca678ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.009305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa5c09-c50f-4766-ac17-1ceaca678ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.049937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ffaa5c09-c50f-4766-ac17-1ceaca678ac7 None None] MFA Rules not processed for user `563c6cbc71ee4cabbbb02047766c9af8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:27.051205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ffaa5c09-c50f-4766-ac17-1ceaca678ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:27.083688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:27.119693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 977/1949] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:27.124334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a21bdc-21fd-434d-9ac3-0ff30058819b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.142661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a21bdc-21fd-434d-9ac3-0ff30058819b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.148847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad2db7-7331-4c1a-a3f8-e3999d27dec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.149472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad2db7-7331-4c1a-a3f8-e3999d27dec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.149749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad2db7-7331-4c1a-a3f8-e3999d27dec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.150015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad2db7-7331-4c1a-a3f8-e3999d27dec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.150713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad2db7-7331-4c1a-a3f8-e3999d27dec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.152625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad2db7-7331-4c1a-a3f8-e3999d27dec8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.195482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 973/1950] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:27 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:27.196365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa5c09-c50f-4766-ac17-1ceaca678ac7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.208775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa5c09-c50f-4766-ac17-1ceaca678ac7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.292492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31967d-16e7-440a-90d9-5dbee71ab627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.293178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31967d-16e7-440a-90d9-5dbee71ab627 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.294113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31967d-16e7-440a-90d9-5dbee71ab627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.294113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31967d-16e7-440a-90d9-5dbee71ab627 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.294679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31967d-16e7-440a-90d9-5dbee71ab627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.296827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31967d-16e7-440a-90d9-5dbee71ab627 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.333661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 978/1951] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:27 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:31:27.335075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad2db7-7331-4c1a-a3f8-e3999d27dec8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.346439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad2db7-7331-4c1a-a3f8-e3999d27dec8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.351267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f34d4-d627-446d-b0e1-356e6e40cdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.351954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f34d4-d627-446d-b0e1-356e6e40cdda admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.352449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f34d4-d627-446d-b0e1-356e6e40cdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.352790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f34d4-d627-446d-b0e1-356e6e40cdda admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.353539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f34d4-d627-446d-b0e1-356e6e40cdda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.355306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f34d4-d627-446d-b0e1-356e6e40cdda admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.367275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 974/1952] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:27.371657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31967d-16e7-440a-90d9-5dbee71ab627 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.382985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31967d-16e7-440a-90d9-5dbee71ab627 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.388424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f224d-beb1-4854-bf1a-ed982a413e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.389235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f224d-beb1-4854-bf1a-ed982a413e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.389475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f224d-beb1-4854-bf1a-ed982a413e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.389739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f224d-beb1-4854-bf1a-ed982a413e6b admin admin] PATH_INFO: `/v3/projects/d1329ce0b0f347b7bd4f75f58fbb3a8f/users/5745c947c5bf4f058f3fa7a205118844/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.407028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f224d-beb1-4854-bf1a-ed982a413e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1329ce0b0f347b7bd4f75f58fbb3a8f, user_id=5745c947c5bf4f058f3fa7a205118844, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.409521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f224d-beb1-4854-bf1a-ed982a413e6b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.436402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 979/1953] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:27 2022] PUT /identity/v3/projects/d1329ce0b0f347b7bd4f75f58fbb3a8f/users/5745c947c5bf4f058f3fa7a205118844/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:27.440293 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f34d4-d627-446d-b0e1-356e6e40cdda admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.452209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f34d4-d627-446d-b0e1-356e6e40cdda admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.507727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:27.518701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36ab94-a4c7-42ba-ac6c-2ff774f2989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.519553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36ab94-a4c7-42ba-ac6c-2ff774f2989c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.520226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36ab94-a4c7-42ba-ac6c-2ff774f2989c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.520494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36ab94-a4c7-42ba-ac6c-2ff774f2989c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.521138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36ab94-a4c7-42ba-ac6c-2ff774f2989c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.522822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36ab94-a4c7-42ba-ac6c-2ff774f2989c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.532933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 975/1954] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:31:27.536990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f224d-beb1-4854-bf1a-ed982a413e6b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.547472 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f224d-beb1-4854-bf1a-ed982a413e6b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.554894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aafbe9-1ef2-4dd9-a12f-44f8e049033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.555577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aafbe9-1ef2-4dd9-a12f-44f8e049033e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.555852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aafbe9-1ef2-4dd9-a12f-44f8e049033e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.556120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aafbe9-1ef2-4dd9-a12f-44f8e049033e admin admin] PATH_INFO: `/v3/projects/d1329ce0b0f347b7bd4f75f58fbb3a8f/users/5745c947c5bf4f058f3fa7a205118844/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.558232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aafbe9-1ef2-4dd9-a12f-44f8e049033e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1329ce0b0f347b7bd4f75f58fbb3a8f, user_id=5745c947c5bf4f058f3fa7a205118844, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.560923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aafbe9-1ef2-4dd9-a12f-44f8e049033e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.572715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 980/1955] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:27 2022] PUT /identity/v3/projects/d1329ce0b0f347b7bd4f75f58fbb3a8f/users/5745c947c5bf4f058f3fa7a205118844/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:31.174597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cdcfd-799a-4329-9aa1-3a1745bfa98d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:31.175156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cdcfd-799a-4329-9aa1-3a1745bfa98d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:31.175156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cdcfd-799a-4329-9aa1-3a1745bfa98d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:31.176512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 976/1956] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:31:31.189766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb582f8-4abf-428a-aded-af6d954d58ef None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:31.190397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb582f8-4abf-428a-aded-af6d954d58ef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:31.190397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb582f8-4abf-428a-aded-af6d954d58ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:31.242973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-acb582f8-4abf-428a-aded-af6d954d58ef None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:31.244203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-acb582f8-4abf-428a-aded-af6d954d58ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:31.368207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 981/1957] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:31:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:31.380608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cdcfd-799a-4329-9aa1-3a1745bfa98d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:31.383012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ad5cdcfd-799a-4329-9aa1-3a1745bfa98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:31.493724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cdcfd-799a-4329-9aa1-3a1745bfa98d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:31.510559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d7b34-7a6f-41dc-8811-72d23f5a21ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fd09151e284f639c93699b0d26db05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:31.511543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d7b34-7a6f-41dc-8811-72d23f5a21ea service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:31.512028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d7b34-7a6f-41dc-8811-72d23f5a21ea service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:31.512502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d7b34-7a6f-41dc-8811-72d23f5a21ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:31.527738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d7b34-7a6f-41dc-8811-72d23f5a21ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:31.530942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d7b34-7a6f-41dc-8811-72d23f5a21ea service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:31.547856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 977/1958] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:31:31.647359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fe627-6642-40b7-a46a-52182e10bfe1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:31.647665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fe627-6642-40b7-a46a-52182e10bfe1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:31.647958 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fe627-6642-40b7-a46a-52182e10bfe1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:31.649343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 982/1959] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:31:31.657338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4498a0-6917-444d-a201-f04b3005d2fe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:31.657625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4498a0-6917-444d-a201-f04b3005d2fe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:31.657899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4498a0-6917-444d-a201-f04b3005d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:31.695353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1f4498a0-6917-444d-a201-f04b3005d2fe None None] MFA Rules not processed for user `c4ad0e5567bd452eadd14ed6ab1edc21`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:31.696606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f4498a0-6917-444d-a201-f04b3005d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:31.824130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 978/1960] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:31:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:31.864117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fe627-6642-40b7-a46a-52182e10bfe1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:31.891694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fe627-6642-40b7-a46a-52182e10bfe1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:31.904331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c222b9-5898-4f15-8364-dd1cfd8adc48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:31.904968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c222b9-5898-4f15-8364-dd1cfd8adc48 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:31.905245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c222b9-5898-4f15-8364-dd1cfd8adc48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:31.905537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c222b9-5898-4f15-8364-dd1cfd8adc48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:31.920876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c222b9-5898-4f15-8364-dd1cfd8adc48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:31.924669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c222b9-5898-4f15-8364-dd1cfd8adc48 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:31.940913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 983/1961] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:32.231975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f22c79-512d-40e7-bb3f-68a2b33c43d5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:32.233162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f22c79-512d-40e7-bb3f-68a2b33c43d5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:32.233506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f22c79-512d-40e7-bb3f-68a2b33c43d5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:32.234964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 979/1962] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:31:32.242894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd42585-7f4a-4bac-afb1-c29bce14dedf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:32.243776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd42585-7f4a-4bac-afb1-c29bce14dedf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:32.244217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd42585-7f4a-4bac-afb1-c29bce14dedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:32.256250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:32.291675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1dd42585-7f4a-4bac-afb1-c29bce14dedf None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:32.298519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1dd42585-7f4a-4bac-afb1-c29bce14dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:32.335218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 984/1963] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:31:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:32.379572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f22c79-512d-40e7-bb3f-68a2b33c43d5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:32.401926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f22c79-512d-40e7-bb3f-68a2b33c43d5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:32.411790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847bf92-53d2-44ea-ab95-ecc106143985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:32.413287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847bf92-53d2-44ea-ab95-ecc106143985 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:32.413885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847bf92-53d2-44ea-ab95-ecc106143985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:32.415202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847bf92-53d2-44ea-ab95-ecc106143985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:32.493574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847bf92-53d2-44ea-ab95-ecc106143985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:32.495739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847bf92-53d2-44ea-ab95-ecc106143985 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:32.521887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 980/1964] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:31:33.783336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519e751-b086-4382-ac12-cd9b9c87ed38 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:33.783899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519e751-b086-4382-ac12-cd9b9c87ed38 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:33.783899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519e751-b086-4382-ac12-cd9b9c87ed38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:33.814390 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9519e751-b086-4382-ac12-cd9b9c87ed38 None None] MFA Rules not processed for user `5745c947c5bf4f058f3fa7a205118844`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:33.815629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9519e751-b086-4382-ac12-cd9b9c87ed38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:33.887392 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 985/1965] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:33.899776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847bf92-53d2-44ea-ab95-ecc106143985 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:33.913722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847bf92-53d2-44ea-ab95-ecc106143985 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:33.918653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fd09151e284f639c93699b0d26db05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:33.919443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:33.920254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:33.920254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:33.989888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:33.991374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:34.001152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 981/1966] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:34.162518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc32fd-c185-4b45-bcfe-c91de19f145d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:34.162813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc32fd-c185-4b45-bcfe-c91de19f145d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:34.163090 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc32fd-c185-4b45-bcfe-c91de19f145d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:34.183333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7dcc32fd-c185-4b45-bcfe-c91de19f145d None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:34.184717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7dcc32fd-c185-4b45-bcfe-c91de19f145d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:34.255629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 986/1967] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:31:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:34.268134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:34.269796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:34.303146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:34.356478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a780-2527-452a-87f7-4ea0c18823af service glance] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:34.361874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c6b70-80ec-46e8-bcf4-546f06005159 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:34.362521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6b70-80ec-46e8-bcf4-546f06005159 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:34.362791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6b70-80ec-46e8-bcf4-546f06005159 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:34.363055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6b70-80ec-46e8-bcf4-546f06005159 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:34.372229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c6b70-80ec-46e8-bcf4-546f06005159 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:34.373945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c6b70-80ec-46e8-bcf4-546f06005159 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:34.382685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 982/1968] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:31:38.410012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491e548-2542-4f4d-b9a4-98cb8968ef22 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:38.410989 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491e548-2542-4f4d-b9a4-98cb8968ef22 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:38.411414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491e548-2542-4f4d-b9a4-98cb8968ef22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:38.436638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e491e548-2542-4f4d-b9a4-98cb8968ef22 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:38.438418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e491e548-2542-4f4d-b9a4-98cb8968ef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:38.456067 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 987/1969] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:31:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:38.469777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c6b70-80ec-46e8-bcf4-546f06005159 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:38.483779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c6b70-80ec-46e8-bcf4-546f06005159 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:38.494371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6c9f8-d8c3-4635-9b26-54e6df3dbe5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:38.494995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6c9f8-d8c3-4635-9b26-54e6df3dbe5b service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:38.495683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6c9f8-d8c3-4635-9b26-54e6df3dbe5b service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:38.495959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6c9f8-d8c3-4635-9b26-54e6df3dbe5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:38.506063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6c9f8-d8c3-4635-9b26-54e6df3dbe5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:38.507840 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6c9f8-d8c3-4635-9b26-54e6df3dbe5b service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:38.518498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 983/1970] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:31:45.115660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8593c-7790-432a-a442-cb4658eebe55 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:45.116261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8593c-7790-432a-a442-cb4658eebe55 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:45.116261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8593c-7790-432a-a442-cb4658eebe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:45.137561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f1c8593c-7790-432a-a442-cb4658eebe55 None None] MFA Rules not processed for user `563c6cbc71ee4cabbbb02047766c9af8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:45.138896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f1c8593c-7790-432a-a442-cb4658eebe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:45.147065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 988/1971] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:31:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:48.514703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=27 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:55.131447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=67 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:57.872909 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2ec61-4f40-431b-9c35-fed7fccebbc7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:57.873479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2ec61-4f40-431b-9c35-fed7fccebbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:57.873725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2ec61-4f40-431b-9c35-fed7fccebbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:57.897718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6cd2ec61-4f40-431b-9c35-fed7fccebbc7 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:57.899037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6cd2ec61-4f40-431b-9c35-fed7fccebbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:57.907376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 984/1972] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:31:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:57.925755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8593c-7790-432a-a442-cb4658eebe55 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:57.943974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8593c-7790-432a-a442-cb4658eebe55 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:57.949592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78555c1f-30b3-4911-b39a-18e22d52ec24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:57.950487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78555c1f-30b3-4911-b39a-18e22d52ec24 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:57.950807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78555c1f-30b3-4911-b39a-18e22d52ec24 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:57.951085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78555c1f-30b3-4911-b39a-18e22d52ec24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:57.960361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78555c1f-30b3-4911-b39a-18e22d52ec24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:57.962660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78555c1f-30b3-4911-b39a-18e22d52ec24 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:57.973861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 989/1973] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:31:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:32:07.891955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:07.969670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:22.081280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55fecd-1921-46a1-99b9-5285f958bec4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:22.087556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55fecd-1921-46a1-99b9-5285f958bec4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:22.088543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55fecd-1921-46a1-99b9-5285f958bec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:22.110006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cc55fecd-1921-46a1-99b9-5285f958bec4 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:22.111341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc55fecd-1921-46a1-99b9-5285f958bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:22.119710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 985/1974] 10.222.0.25 () {58 vars in 967 bytes} [Wed Nov 2 07:32:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:22.142621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78555c1f-30b3-4911-b39a-18e22d52ec24 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:22.153339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78555c1f-30b3-4911-b39a-18e22d52ec24 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:22.158666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee43e8-aca9-4e19-bddd-bdb906f01d51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:22.159256 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee43e8-aca9-4e19-bddd-bdb906f01d51 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:22.159532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee43e8-aca9-4e19-bddd-bdb906f01d51 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:22.159832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee43e8-aca9-4e19-bddd-bdb906f01d51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:22.167914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee43e8-aca9-4e19-bddd-bdb906f01d51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:22.169808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee43e8-aca9-4e19-bddd-bdb906f01d51 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:22.177650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 990/1975] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:32:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:32:32.104565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:32.174846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:36.009647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55fecd-1921-46a1-99b9-5285f958bec4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:36.011093 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc55fecd-1921-46a1-99b9-5285f958bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:36.024104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55fecd-1921-46a1-99b9-5285f958bec4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:36.028636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6136d1-c13f-4c66-be63-e319a44047de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:36.029360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6136d1-c13f-4c66-be63-e319a44047de service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:36.029651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6136d1-c13f-4c66-be63-e319a44047de service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:36.029947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6136d1-c13f-4c66-be63-e319a44047de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:36.039982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6136d1-c13f-4c66-be63-e319a44047de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:36.041423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6136d1-c13f-4c66-be63-e319a44047de service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:36.051880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 986/1976] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:42.323477 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee43e8-aca9-4e19-bddd-bdb906f01d51 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:42.333200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee43e8-aca9-4e19-bddd-bdb906f01d51 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:42.337065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:42.337651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:42.337919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:42.338222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:42.340081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:42.359413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:42.360847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:42.373006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 991/1977] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:43.558851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6136d1-c13f-4c66-be63-e319a44047de service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:43.568835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6136d1-c13f-4c66-be63-e319a44047de service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:43.573868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:43.574498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:43.574776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:43.575043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:43.577360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:43.592168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:43.593596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:43.605118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 987/1978] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:52.367117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:53.600144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=12 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:54.578906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:54.580588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:54.594688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b132b-c9ab-4cde-8458-1014822eeaf0 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:54.601459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b9779-a159-4dbe-8d57-fc3417c2cb2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:54.602048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b9779-a159-4dbe-8d57-fc3417c2cb2d service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:54.602358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b9779-a159-4dbe-8d57-fc3417c2cb2d service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:54.602630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b9779-a159-4dbe-8d57-fc3417c2cb2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:54.604404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3b2b9779-a159-4dbe-8d57-fc3417c2cb2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:54.668736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b9779-a159-4dbe-8d57-fc3417c2cb2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:54.670602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b9779-a159-4dbe-8d57-fc3417c2cb2d service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:54.679827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 992/1979] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:04.675755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=28 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:10.101754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:10.116539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7475ba-e4f9-4550-a227-73b61e151502 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:10.122579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:10.123173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:10.123449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:10.123718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:10.125576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:10.185497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:10.187187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:10.198430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 988/1980] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:20.192549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=28 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:25.782170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9be880-fff8-4628-ba3c-520573098c25 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:25.783181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9be880-fff8-4628-ba3c-520573098c25 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:25.783181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9be880-fff8-4628-ba3c-520573098c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:25.814675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8d9be880-fff8-4628-ba3c-520573098c25 None None] MFA Rules not processed for user `0e8717d368d44102bea0eb82fbee8a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:33:25.816847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8d9be880-fff8-4628-ba3c-520573098c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:25.825524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 993/1981] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:33:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:33:26.199675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:26.209793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4716b96-52be-4788-87ae-bb2fb81ec1a8 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:26.214049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:26.214809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:26.215089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:26.215354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:26.217768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:26.229322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:26.231152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:26.241074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 989/1982] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:26.591923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9be880-fff8-4628-ba3c-520573098c25 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:26.593734 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8d9be880-fff8-4628-ba3c-520573098c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:26.668165 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9be880-fff8-4628-ba3c-520573098c25 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:26.673612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:26.674265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:26.674517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:26.674790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:26.676837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:26.689117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:26.690717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:26.701120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 994/1983] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:28.984230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:28.993423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ce260-f535-4e74-b3a4-3941f3755b98 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:28.996657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:28.997252 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:28.997530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:28.997799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:28.999822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:29.009504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:29.010959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:29.019192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 990/1984] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:29.680605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:29.697160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4e52f-3de3-4acb-809a-3a1e0abcabbc service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:29.704460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3438fe0-8b01-4924-af9f-21de8648f36c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:29.705085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3438fe0-8b01-4924-af9f-21de8648f36c service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:29.705355 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3438fe0-8b01-4924-af9f-21de8648f36c service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:29.705628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3438fe0-8b01-4924-af9f-21de8648f36c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:29.707538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a3438fe0-8b01-4924-af9f-21de8648f36c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:29.719902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3438fe0-8b01-4924-af9f-21de8648f36c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:29.721824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3438fe0-8b01-4924-af9f-21de8648f36c service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:29.733730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 995/1985] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:39.015566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=12 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:39.728622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=41 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:39.902405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:39.916391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0d5a-3f70-4237-b41e-4cb19c94a10d service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:39.922763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:39.923595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:39.923984 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:39.924123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:39.926878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:39.994655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:39.996268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:40.007392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 991/1986] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:48.676276 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0126e-8773-446f-a4fe-50cbb438e15c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:48.676800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0126e-8773-446f-a4fe-50cbb438e15c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:48.676896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0126e-8773-446f-a4fe-50cbb438e15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:48.696959 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-96e0126e-8773-446f-a4fe-50cbb438e15c None None] MFA Rules not processed for user `006ae080f2884719bd4ddabe815d0d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:33:48.698180 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-96e0126e-8773-446f-a4fe-50cbb438e15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:48.705482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 996/1987] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:33:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:33:50.002820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=28 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:57.381506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:57.393536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d30f9-bf39-46ea-b74b-9a8b8dec23bc service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:57.400332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:57.400929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:57.401201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:57.401466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:57.403448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:57.416816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:57.418472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:57.429958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 992/1988] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:58.691474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:03.466017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:03.467828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38116f-4375-45e2-9c8b-155cf5ff622b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:03.468254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38116f-4375-45e2-9c8b-155cf5ff622b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:03.468389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38116f-4375-45e2-9c8b-155cf5ff622b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:03.483460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4679b9-cd64-436c-af24-38b67619c5c8 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:03.499186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6b38116f-4375-45e2-9c8b-155cf5ff622b None None] MFA Rules not processed for user `08c9e3cb6c164e72b0d55a2724469440`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:03.500594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b38116f-4375-45e2-9c8b-155cf5ff622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:03.515022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71760d0c-f50c-46c1-acd0-e333ea9519e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:03.515669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71760d0c-f50c-46c1-acd0-e333ea9519e8 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:03.515955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71760d0c-f50c-46c1-acd0-e333ea9519e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:03.516232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71760d0c-f50c-46c1-acd0-e333ea9519e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:03.521150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-71760d0c-f50c-46c1-acd0-e333ea9519e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:03.534582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71760d0c-f50c-46c1-acd0-e333ea9519e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:03.536201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71760d0c-f50c-46c1-acd0-e333ea9519e8 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:03.548890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 993/1989] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:03.570132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 997/1990] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:34:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:13.516385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc7ad1-afec-4a8d-9cd1-84cb4bad5d85 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:13.516977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc7ad1-afec-4a8d-9cd1-84cb4bad5d85 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:13.516977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc7ad1-afec-4a8d-9cd1-84cb4bad5d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:13.537341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2fcc7ad1-afec-4a8d-9cd1-84cb4bad5d85 None None] MFA Rules not processed for user `8f151dc283e8498f961b49320130f770`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:13.538698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2fcc7ad1-afec-4a8d-9cd1-84cb4bad5d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:13.547175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 994/1991] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:34:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:13.557939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=25 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:13.884633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38116f-4375-45e2-9c8b-155cf5ff622b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:13.897866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38116f-4375-45e2-9c8b-155cf5ff622b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:13.902072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53f08-35b9-400b-967f-733020b029cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:13.902966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53f08-35b9-400b-967f-733020b029cb service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:13.903357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53f08-35b9-400b-967f-733020b029cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:13.903769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53f08-35b9-400b-967f-733020b029cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:13.907014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-91e53f08-35b9-400b-967f-733020b029cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:13.919916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53f08-35b9-400b-967f-733020b029cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:13.921416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53f08-35b9-400b-967f-733020b029cb service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:13.930121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 998/1992] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:23.532211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=15 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:23.926630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:20.483391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc7ad1-afec-4a8d-9cd1-84cb4bad5d85 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:20.485147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2fcc7ad1-afec-4a8d-9cd1-84cb4bad5d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:20.563759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc7ad1-afec-4a8d-9cd1-84cb4bad5d85 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:20.567867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84023f6-83c0-4028-aec7-535ed939719c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:20.568702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84023f6-83c0-4028-aec7-535ed939719c service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:20.569142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84023f6-83c0-4028-aec7-535ed939719c service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:20.569543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84023f6-83c0-4028-aec7-535ed939719c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:20.571713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b84023f6-83c0-4028-aec7-535ed939719c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:20.584676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84023f6-83c0-4028-aec7-535ed939719c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:20.586446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84023f6-83c0-4028-aec7-535ed939719c service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:20.602390 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 995/1993] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:24.525867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898da8-0c78-4071-b111-821e31b8d476 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:24.526492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898da8-0c78-4071-b111-821e31b8d476 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:24.526492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898da8-0c78-4071-b111-821e31b8d476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:24.547821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-01898da8-0c78-4071-b111-821e31b8d476 None None] MFA Rules not processed for user `0e8717d368d44102bea0eb82fbee8a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:24.549007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-01898da8-0c78-4071-b111-821e31b8d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:24.557479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 999/1994] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:35:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:30.054593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51112a-d10f-4316-9815-5084147e14c9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:30.055428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51112a-d10f-4316-9815-5084147e14c9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:30.056032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51112a-d10f-4316-9815-5084147e14c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:30.095165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9c51112a-d10f-4316-9815-5084147e14c9 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:30.097025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9c51112a-d10f-4316-9815-5084147e14c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:30.105355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 996/1995] 10.222.0.25 () {58 vars in 967 bytes} [Wed Nov 2 07:35:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:30.159826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01898da8-0c78-4071-b111-821e31b8d476 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:30.172638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01898da8-0c78-4071-b111-821e31b8d476 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:30.178499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3810437-70d7-460c-bf19-e761de73a28b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:30.179202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3810437-70d7-460c-bf19-e761de73a28b service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:30.179546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3810437-70d7-460c-bf19-e761de73a28b service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:30.179882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3810437-70d7-460c-bf19-e761de73a28b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:30.191491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3810437-70d7-460c-bf19-e761de73a28b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:30.196141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3810437-70d7-460c-bf19-e761de73a28b service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:30.219344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1000/1996] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:35:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:35:34.269578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51112a-d10f-4316-9815-5084147e14c9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:34.293535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51112a-d10f-4316-9815-5084147e14c9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:34.302771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:34.304351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:34.304967 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:34.306044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:34.312160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:34.361687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:34.363771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:34.389735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 997/1997] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:39.003853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3810437-70d7-460c-bf19-e761de73a28b service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.066542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3810437-70d7-460c-bf19-e761de73a28b service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.070994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.071651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.071925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.072200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] PATH_INFO: `/v3/users/563c6cbc71ee4cabbbb02047766c9af8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.073519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563c6cbc71ee4cabbbb02047766c9af8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.075538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.137107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563c6cbc71ee4cabbbb02047766c9af8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.137899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563c6cbc71ee4cabbbb02047766c9af8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.151978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563c6cbc71ee4cabbbb02047766c9af8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.159983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563c6cbc71ee4cabbbb02047766c9af8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.160691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563c6cbc71ee4cabbbb02047766c9af8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.175483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1001/1998] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:39 2022] DELETE /identity/v3/users/563c6cbc71ee4cabbbb02047766c9af8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:39.363641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.375557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fec26f-5583-48bf-b4c2-5db86818fd84 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.441217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.442021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.442385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.442721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] PATH_INFO: `/v3/projects/c513abe395e34a08ba8fa7d59a5aae74` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.443950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c513abe395e34a08ba8fa7d59a5aae74)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.446165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.496436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c513abe395e34a08ba8fa7d59a5aae74', 'request_id': 'req-66031c7b-f3f2-4f42-97bc-75352855f95b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.502561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c513abe395e34a08ba8fa7d59a5aae74', 'request_id': 'req-66031c7b-f3f2-4f42-97bc-75352855f95b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.510717 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c513abe395e34a08ba8fa7d59a5aae74', 'request_id': 'req-66031c7b-f3f2-4f42-97bc-75352855f95b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.523304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] The token cache is being invalidate because project c513abe395e34a08ba8fa7d59a5aae74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:39.523749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.524751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.528801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 998/1999] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:39 2022] DELETE /identity/v3/projects/c513abe395e34a08ba8fa7d59a5aae74 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:39.533176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.534965 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:39.549923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292475e-d300-4c08-bb77-713edebb4975 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.633042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.633713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.634025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.634321 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] PATH_INFO: `/v3/users/5745c947c5bf4f058f3fa7a205118844` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.640727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5745c947c5bf4f058f3fa7a205118844)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.643190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.644538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:39.718676 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5745c947c5bf4f058f3fa7a205118844'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.719446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5745c947c5bf4f058f3fa7a205118844'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.733178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5745c947c5bf4f058f3fa7a205118844'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.741048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5745c947c5bf4f058f3fa7a205118844'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.741756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5745c947c5bf4f058f3fa7a205118844'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.763351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1002/2000] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:39 2022] DELETE /identity/v3/users/5745c947c5bf4f058f3fa7a205118844 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:40.016897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.018398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:40.032324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66031c7b-f3f2-4f42-97bc-75352855f95b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.044655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:40.118816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.119888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.120355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.120786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] PATH_INFO: `/v3/projects/d1329ce0b0f347b7bd4f75f58fbb3a8f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.122143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1329ce0b0f347b7bd4f75f58fbb3a8f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.124226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.186938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1329ce0b0f347b7bd4f75f58fbb3a8f', 'request_id': 'req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:40.193182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1329ce0b0f347b7bd4f75f58fbb3a8f', 'request_id': 'req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:40.201584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1329ce0b0f347b7bd4f75f58fbb3a8f', 'request_id': 'req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:40.212353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] The token cache is being invalidate because project d1329ce0b0f347b7bd4f75f58fbb3a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:40.212637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:40.213304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:40.214920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 999/2001] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:40 2022] DELETE /identity/v3/projects/d1329ce0b0f347b7bd4f75f58fbb3a8f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:44.805474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:44.807153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:44.905001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a2614-2c83-4fc4-b5ac-4297c0f54361 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:44.968193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:44.968810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:44.969088 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:44.969358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:44.971813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:44.994045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:45.085716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:45.087181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:45.108282 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1003/2002] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:48.390174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:48.391650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:48.491785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:48.493725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:48.511021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f0124-bce8-4240-9657-bd952ac57fe3 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:48.517108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:48.517108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.517514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.517606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] PATH_INFO: `/v3/users/08c9e3cb6c164e72b0d55a2724469440` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.518796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c9e3cb6c164e72b0d55a2724469440)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:48.533488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:48.563971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:48.569313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0f8c8-00e3-4548-be9a-c2ac6c5eb259 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:48.579123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:48.579931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.580375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.580671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.588541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c9e3cb6c164e72b0d55a2724469440'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.589420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c9e3cb6c164e72b0d55a2724469440'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.591756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:48.603832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c9e3cb6c164e72b0d55a2724469440'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.605913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:48.607722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:48.611921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c9e3cb6c164e72b0d55a2724469440'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.612639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e844842e-fb28-4972-b109-0b199f10e979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c9e3cb6c164e72b0d55a2724469440'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.619690 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1000/2003] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:48.632272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1004/2004] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:48 2022] DELETE /identity/v3/users/08c9e3cb6c164e72b0d55a2724469440 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:48.836453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:48.848905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1d8e1-ca2a-4dbe-89b4-6b14abe4c030 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:48.907759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:48.908386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.908665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.908928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] PATH_INFO: `/v3/projects/0d2056a08a324aed92eca4be58e888d6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.910171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d2056a08a324aed92eca4be58e888d6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:48.912122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:48.960558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d2056a08a324aed92eca4be58e888d6', 'request_id': 'req-d17fa005-85fe-4e7f-9532-72a5a12d657c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.967741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d2056a08a324aed92eca4be58e888d6', 'request_id': 'req-d17fa005-85fe-4e7f-9532-72a5a12d657c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.978123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d2056a08a324aed92eca4be58e888d6', 'request_id': 'req-d17fa005-85fe-4e7f-9532-72a5a12d657c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.989094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] The token cache is being invalidate because project 0d2056a08a324aed92eca4be58e888d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:48.989377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.990008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d17fa005-85fe-4e7f-9532-72a5a12d657c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:48.991537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1001/2005] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:48 2022] DELETE /identity/v3/projects/0d2056a08a324aed92eca4be58e888d6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:48.998020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97416661-7ad1-4621-be21-530b41f29bd0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.998423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97416661-7ad1-4621-be21-530b41f29bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.998830 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97416661-7ad1-4621-be21-530b41f29bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.063730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-97416661-7ad1-4621-be21-530b41f29bd0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:49.064938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-97416661-7ad1-4621-be21-530b41f29bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:49.069346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:49.127500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1005/2006] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:49.139132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0fc36-e83e-4fb5-86dc-3490494e4749 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.139415 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0fc36-e83e-4fb5-86dc-3490494e4749 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.139688 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0fc36-e83e-4fb5-86dc-3490494e4749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.159697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-74c0fc36-e83e-4fb5-86dc-3490494e4749 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:49.160887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-74c0fc36-e83e-4fb5-86dc-3490494e4749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:49.169561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1002/2007] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:49.176483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97416661-7ad1-4621-be21-530b41f29bd0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:49.192751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97416661-7ad1-4621-be21-530b41f29bd0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:49.200621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:49.201244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.201523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.201798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.202504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:49.202811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:49.204411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:49.216249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1006/2008] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:35:49.221596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0fc36-e83e-4fb5-86dc-3490494e4749 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:49.231537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0fc36-e83e-4fb5-86dc-3490494e4749 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:49.237537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865cb91-bfe0-4f83-b148-5120c6a9f7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:49.238177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865cb91-bfe0-4f83-b148-5120c6a9f7b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.238432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865cb91-bfe0-4f83-b148-5120c6a9f7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.238721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865cb91-bfe0-4f83-b148-5120c6a9f7b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.239397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865cb91-bfe0-4f83-b148-5120c6a9f7b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:49.241307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865cb91-bfe0-4f83-b148-5120c6a9f7b3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:49.276433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1003/2009] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:49 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:49.281468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:49.291620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d58c0-0622-4026-a4b4-4ca356bfa31a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:49.348193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f70d81-5ae7-4d65-bb10-6d8a2cfde253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:49.348899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f70d81-5ae7-4d65-bb10-6d8a2cfde253 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.349195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f70d81-5ae7-4d65-bb10-6d8a2cfde253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.349464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f70d81-5ae7-4d65-bb10-6d8a2cfde253 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.350175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f70d81-5ae7-4d65-bb10-6d8a2cfde253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:49.352165 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f70d81-5ae7-4d65-bb10-6d8a2cfde253 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:49.374457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1007/2010] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:49 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:49.378813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865cb91-bfe0-4f83-b148-5120c6a9f7b3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:49.389141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865cb91-bfe0-4f83-b148-5120c6a9f7b3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:49.393931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c16bfc-413d-4f2f-a27b-51bf68175d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:49.394552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16bfc-413d-4f2f-a27b-51bf68175d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.394812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16bfc-413d-4f2f-a27b-51bf68175d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.395081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16bfc-413d-4f2f-a27b-51bf68175d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.395738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c16bfc-413d-4f2f-a27b-51bf68175d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:49.397321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c16bfc-413d-4f2f-a27b-51bf68175d70 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:49.407895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1004/2011] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:49.411863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f70d81-5ae7-4d65-bb10-6d8a2cfde253 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:49.421976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f70d81-5ae7-4d65-bb10-6d8a2cfde253 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:49.427307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069d067-253e-4bae-9673-148cd790bdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:49.427947 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d067-253e-4bae-9673-148cd790bdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.428221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d067-253e-4bae-9673-148cd790bdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.428490 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d067-253e-4bae-9673-148cd790bdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.429150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069d067-253e-4bae-9673-148cd790bdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:49.430857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069d067-253e-4bae-9673-148cd790bdc6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:49.442395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1008/2012] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:49.446843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c16bfc-413d-4f2f-a27b-51bf68175d70 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:49.458230 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c16bfc-413d-4f2f-a27b-51bf68175d70 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:49.463330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:49.463983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.464261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.464534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] PATH_INFO: `/v3/projects/a4ee02d79fdd4419a3b6a32ae2c319ca/users/f33d2b2c70e54860be74434e4aee63f7/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.480241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ee02d79fdd4419a3b6a32ae2c319ca, user_id=f33d2b2c70e54860be74434e4aee63f7, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:49.482838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:49.498366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1005/2013] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:49 2022] PUT /identity/v3/projects/a4ee02d79fdd4419a3b6a32ae2c319ca/users/f33d2b2c70e54860be74434e4aee63f7/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:49.508378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4e11-ceb3-4add-8b4b-38bae0614002 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.509023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4e11-ceb3-4add-8b4b-38bae0614002 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.509259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4e11-ceb3-4add-8b4b-38bae0614002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.539084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c8ed4e11-ceb3-4add-8b4b-38bae0614002 None None] MFA Rules not processed for user `f33d2b2c70e54860be74434e4aee63f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:49.540354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c8ed4e11-ceb3-4add-8b4b-38bae0614002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:49.609616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1009/2014] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:35:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:49.621199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:49.623162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:49.642559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:49.685620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb48a8-1469-46e6-a468-b1727e664608 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:49.691179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:49.692080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.692402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.692670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.701711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:49.703432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:49.712596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1006/2015] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:51.569357 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a3a8f-b7ed-44f5-8cc2-b19bd3d79029 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:51.569839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a3a8f-b7ed-44f5-8cc2-b19bd3d79029 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:51.569926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a3a8f-b7ed-44f5-8cc2-b19bd3d79029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:51.576722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:51.604748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d92a3a8f-b7ed-44f5-8cc2-b19bd3d79029 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:51.606672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d92a3a8f-b7ed-44f5-8cc2-b19bd3d79029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:51.678821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1010/2016] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:35:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:51.691688 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:51.693290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:51.768112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eb016-39ec-4b0e-9f85-21fb867e2282 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:51.774267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0dec3-ea7b-4a15-a62d-b701c28e79eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:51.774883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0dec3-ea7b-4a15-a62d-b701c28e79eb service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:51.775159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0dec3-ea7b-4a15-a62d-b701c28e79eb service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:51.775439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0dec3-ea7b-4a15-a62d-b701c28e79eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:51.784312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0dec3-ea7b-4a15-a62d-b701c28e79eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:51.785986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0dec3-ea7b-4a15-a62d-b701c28e79eb service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:51.794462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1007/2017] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:35:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:35:58.721418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a3a8f-b7ed-44f5-8cc2-b19bd3d79029 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.853086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a3a8f-b7ed-44f5-8cc2-b19bd3d79029 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.858347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.859333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.859577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.860886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] PATH_INFO: `/v3/users/f33d2b2c70e54860be74434e4aee63f7` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.866217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f33d2b2c70e54860be74434e4aee63f7)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.869234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.970795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f33d2b2c70e54860be74434e4aee63f7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:58.972034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f33d2b2c70e54860be74434e4aee63f7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:58.991401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f33d2b2c70e54860be74434e4aee63f7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:59.014017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f33d2b2c70e54860be74434e4aee63f7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:59.015445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f33d2b2c70e54860be74434e4aee63f7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:59.040090 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1011/2018] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:58 2022] DELETE /identity/v3/users/f33d2b2c70e54860be74434e4aee63f7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:59.041792 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52960212-fcbd-4d1b-b32a-826fb468ce04 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.042639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52960212-fcbd-4d1b-b32a-826fb468ce04 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.043056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52960212-fcbd-4d1b-b32a-826fb468ce04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.074124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-52960212-fcbd-4d1b-b32a-826fb468ce04 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:59.075832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52960212-fcbd-4d1b-b32a-826fb468ce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.238631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1008/2019] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:59.245270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:59.247002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.264815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:59.338191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b5e5d-91ac-4d32-bb6f-3c8c824eee1b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:59.343742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef7d7d-35fa-4d4f-8e6c-1a34457833a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:59.344521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7d7d-35fa-4d4f-8e6c-1a34457833a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.344901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7d7d-35fa-4d4f-8e6c-1a34457833a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.345277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7d7d-35fa-4d4f-8e6c-1a34457833a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.357147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef7d7d-35fa-4d4f-8e6c-1a34457833a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:59.358753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef7d7d-35fa-4d4f-8e6c-1a34457833a8 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:59.376365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1012/2020] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:00.049765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52960212-fcbd-4d1b-b32a-826fb468ce04 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:00.051508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52960212-fcbd-4d1b-b32a-826fb468ce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:00.072591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52960212-fcbd-4d1b-b32a-826fb468ce04 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:00.086191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:00.086835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.087972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.088600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] PATH_INFO: `/v3/projects/a4ee02d79fdd4419a3b6a32ae2c319ca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.089997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ee02d79fdd4419a3b6a32ae2c319ca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:00.092051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:00.165721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:00.175285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ee02d79fdd4419a3b6a32ae2c319ca', 'request_id': 'req-7823cca3-93f9-478b-81a6-7716c0a09e50'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:00.185223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ee02d79fdd4419a3b6a32ae2c319ca', 'request_id': 'req-7823cca3-93f9-478b-81a6-7716c0a09e50'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:00.199037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ee02d79fdd4419a3b6a32ae2c319ca', 'request_id': 'req-7823cca3-93f9-478b-81a6-7716c0a09e50'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:00.210843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] The token cache is being invalidate because project a4ee02d79fdd4419a3b6a32ae2c319ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:00.211260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:00.211994 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7823cca3-93f9-478b-81a6-7716c0a09e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:00.213760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1009/2021] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:00 2022] DELETE /identity/v3/projects/a4ee02d79fdd4419a3b6a32ae2c319ca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:00.218843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61823c9-34fc-41e2-8a4d-c11a9a4e643c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.219710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61823c9-34fc-41e2-8a4d-c11a9a4e643c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.220145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61823c9-34fc-41e2-8a4d-c11a9a4e643c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.248221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d61823c9-34fc-41e2-8a4d-c11a9a4e643c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:00.249681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d61823c9-34fc-41e2-8a4d-c11a9a4e643c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:00.332620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1013/2022] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:00.341578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b724d83-f0b0-4486-8abd-1e75ab24267d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.341957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b724d83-f0b0-4486-8abd-1e75ab24267d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.342363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b724d83-f0b0-4486-8abd-1e75ab24267d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.379392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9b724d83-f0b0-4486-8abd-1e75ab24267d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:00.380712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9b724d83-f0b0-4486-8abd-1e75ab24267d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:00.395975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1010/2023] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:00.400408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61823c9-34fc-41e2-8a4d-c11a9a4e643c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:00.412500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61823c9-34fc-41e2-8a4d-c11a9a4e643c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:00.419666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:00.420484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.420760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.421029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.421771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:00.422070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:00.423795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:00.436679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1014/2024] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:00.447618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b724d83-f0b0-4486-8abd-1e75ab24267d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:00.463189 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b724d83-f0b0-4486-8abd-1e75ab24267d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:00.479570 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512b2e-a895-4a1f-ab25-3062d512f8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:00.480708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512b2e-a895-4a1f-ab25-3062d512f8fd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.481165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512b2e-a895-4a1f-ab25-3062d512f8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.481870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512b2e-a895-4a1f-ab25-3062d512f8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.482680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75512b2e-a895-4a1f-ab25-3062d512f8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:00.484663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75512b2e-a895-4a1f-ab25-3062d512f8fd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:00.524892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1011/2025] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:00 2022] POST /identity/v3/projects => generated 403 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:00.529140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:00.540553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e2a10-967d-4643-bcca-6975bb0f4166 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:00.632833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc975b1-53b4-4550-830e-4cab9a3b8601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:00.633468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc975b1-53b4-4550-830e-4cab9a3b8601 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.633746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc975b1-53b4-4550-830e-4cab9a3b8601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.634016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc975b1-53b4-4550-830e-4cab9a3b8601 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.634703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc975b1-53b4-4550-830e-4cab9a3b8601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:00.636680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc975b1-53b4-4550-830e-4cab9a3b8601 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:00.668155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1015/2026] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:00 2022] POST /identity/v3/users => generated 302 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:00.669480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512b2e-a895-4a1f-ab25-3062d512f8fd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:00.681192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512b2e-a895-4a1f-ab25-3062d512f8fd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:00.689311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594208f-73c5-4cbe-8996-fc10cafb5fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:00.689863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594208f-73c5-4cbe-8996-fc10cafb5fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.690162 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594208f-73c5-4cbe-8996-fc10cafb5fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.690432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594208f-73c5-4cbe-8996-fc10cafb5fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.691069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594208f-73c5-4cbe-8996-fc10cafb5fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:00.692669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594208f-73c5-4cbe-8996-fc10cafb5fc8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:00.704188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1012/2027] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:00.707995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc975b1-53b4-4550-830e-4cab9a3b8601 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:00.738213 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc975b1-53b4-4550-830e-4cab9a3b8601 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:00.743934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb0d06-aeb3-46a5-91eb-ebb87a831efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:00.744752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0d06-aeb3-46a5-91eb-ebb87a831efc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.745038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0d06-aeb3-46a5-91eb-ebb87a831efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.745311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0d06-aeb3-46a5-91eb-ebb87a831efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.745986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb0d06-aeb3-46a5-91eb-ebb87a831efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:00.747699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb0d06-aeb3-46a5-91eb-ebb87a831efc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:00.760494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1016/2028] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:00.769665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594208f-73c5-4cbe-8996-fc10cafb5fc8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:00.780169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594208f-73c5-4cbe-8996-fc10cafb5fc8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:00.785783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c71b4-90ac-4fad-bb28-cfb8370a7068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:00.786498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c71b4-90ac-4fad-bb28-cfb8370a7068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.786780 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c71b4-90ac-4fad-bb28-cfb8370a7068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.787055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c71b4-90ac-4fad-bb28-cfb8370a7068 admin admin] PATH_INFO: `/v3/projects/892b1494626a45179257cbac98e5be0f/users/b08503068f3c4bfc848ad6208fe2f558/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.802516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c71b4-90ac-4fad-bb28-cfb8370a7068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892b1494626a45179257cbac98e5be0f, user_id=b08503068f3c4bfc848ad6208fe2f558, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:00.805256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c71b4-90ac-4fad-bb28-cfb8370a7068 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:00.819994 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1013/2029] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:00 2022] PUT /identity/v3/projects/892b1494626a45179257cbac98e5be0f/users/b08503068f3c4bfc848ad6208fe2f558/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:00.830418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f7269-6c9e-4297-9859-52a02f48bf7b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:00.830711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f7269-6c9e-4297-9859-52a02f48bf7b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:00.830992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f7269-6c9e-4297-9859-52a02f48bf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:00.865159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a99f7269-6c9e-4297-9859-52a02f48bf7b None None] MFA Rules not processed for user `b08503068f3c4bfc848ad6208fe2f558`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:00.866515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a99f7269-6c9e-4297-9859-52a02f48bf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:00.876965 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:00.956780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1017/2030] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:00.961613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c71b4-90ac-4fad-bb28-cfb8370a7068 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:01.033802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c71b4-90ac-4fad-bb28-cfb8370a7068 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:01.038725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b4f53-2fcf-4680-b9bc-d4e3e8be8396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:01.039459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b4f53-2fcf-4680-b9bc-d4e3e8be8396 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.039817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b4f53-2fcf-4680-b9bc-d4e3e8be8396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.040179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b4f53-2fcf-4680-b9bc-d4e3e8be8396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.040924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b4f53-2fcf-4680-b9bc-d4e3e8be8396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:01.043054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b4f53-2fcf-4680-b9bc-d4e3e8be8396 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:01.075942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1014/2031] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:00 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:01.079985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f7269-6c9e-4297-9859-52a02f48bf7b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:01.092226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f7269-6c9e-4297-9859-52a02f48bf7b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:01.174453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce278434-601b-45e6-bd26-f9cc595c6de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:01.175130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce278434-601b-45e6-bd26-f9cc595c6de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.175414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce278434-601b-45e6-bd26-f9cc595c6de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.175688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce278434-601b-45e6-bd26-f9cc595c6de6 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.176751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce278434-601b-45e6-bd26-f9cc595c6de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:01.179791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce278434-601b-45e6-bd26-f9cc595c6de6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:01.218360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1018/2032] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:01 2022] POST /identity/v3/users => generated 308 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:01.219499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b4f53-2fcf-4680-b9bc-d4e3e8be8396 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:01.232356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b4f53-2fcf-4680-b9bc-d4e3e8be8396 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:01.237651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c37f0-17bf-47c8-865d-ef75694d6df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:01.238370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c37f0-17bf-47c8-865d-ef75694d6df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.238749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c37f0-17bf-47c8-865d-ef75694d6df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.239078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c37f0-17bf-47c8-865d-ef75694d6df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.239784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c37f0-17bf-47c8-865d-ef75694d6df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:01.241487 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c37f0-17bf-47c8-865d-ef75694d6df0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:01.255215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1015/2033] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:36:01.259242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce278434-601b-45e6-bd26-f9cc595c6de6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:01.273701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce278434-601b-45e6-bd26-f9cc595c6de6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:01.278688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f5ecf-b9d0-40c2-90c4-ad054d1f4695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:01.279447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f5ecf-b9d0-40c2-90c4-ad054d1f4695 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.279825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f5ecf-b9d0-40c2-90c4-ad054d1f4695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.280186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f5ecf-b9d0-40c2-90c4-ad054d1f4695 admin admin] PATH_INFO: `/v3/projects/988cab6db7d64312a95b63bcf036b430/users/7520c2c368be42bfab928d391b229f52/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.296893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f5ecf-b9d0-40c2-90c4-ad054d1f4695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=988cab6db7d64312a95b63bcf036b430, user_id=7520c2c368be42bfab928d391b229f52, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:01.300025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f5ecf-b9d0-40c2-90c4-ad054d1f4695 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:01.315826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1019/2034] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:01 2022] PUT /identity/v3/projects/988cab6db7d64312a95b63bcf036b430/users/7520c2c368be42bfab928d391b229f52/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:01.320256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c37f0-17bf-47c8-865d-ef75694d6df0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:01.331999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c37f0-17bf-47c8-865d-ef75694d6df0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:01.385335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:01.408033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f828b1-7362-4aa5-ba4d-de8f388d37ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:01.408716 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f828b1-7362-4aa5-ba4d-de8f388d37ac admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.408998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f828b1-7362-4aa5-ba4d-de8f388d37ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.409266 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f828b1-7362-4aa5-ba4d-de8f388d37ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.409936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f828b1-7362-4aa5-ba4d-de8f388d37ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:01.411922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f828b1-7362-4aa5-ba4d-de8f388d37ac admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:01.425822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1016/2035] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:01.427054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f5ecf-b9d0-40c2-90c4-ad054d1f4695 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:01.446185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f5ecf-b9d0-40c2-90c4-ad054d1f4695 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:01.451091 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:01.451844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.452193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.452535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] PATH_INFO: `/v3/projects/988cab6db7d64312a95b63bcf036b430/users/7520c2c368be42bfab928d391b229f52/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.454616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=988cab6db7d64312a95b63bcf036b430, user_id=7520c2c368be42bfab928d391b229f52, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:01.457334 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:01.472509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1020/2036] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:01 2022] PUT /identity/v3/projects/988cab6db7d64312a95b63bcf036b430/users/7520c2c368be42bfab928d391b229f52/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:01.483146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6b758-7bbd-42ac-988f-13d062ac10bd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.483464 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6b758-7bbd-42ac-988f-13d062ac10bd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.483743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6b758-7bbd-42ac-988f-13d062ac10bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.514966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-54f6b758-7bbd-42ac-988f-13d062ac10bd None None] MFA Rules not processed for user `7520c2c368be42bfab928d391b229f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:01.516280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-54f6b758-7bbd-42ac-988f-13d062ac10bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:01.591137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1017/2037] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:01 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:01.602911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:01.604612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:01.667025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212cb41-b119-4a27-accf-c5ac8e713ee4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:01.670697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190e22-0e02-4bd1-91a8-96919822a265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:01.671310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190e22-0e02-4bd1-91a8-96919822a265 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.671736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190e22-0e02-4bd1-91a8-96919822a265 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.672009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190e22-0e02-4bd1-91a8-96919822a265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.681201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4190e22-0e02-4bd1-91a8-96919822a265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:01.682659 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4190e22-0e02-4bd1-91a8-96919822a265 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:01.692424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1021/2038] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:01 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:01.858047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6b758-7bbd-42ac-988f-13d062ac10bd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:01.928041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6b758-7bbd-42ac-988f-13d062ac10bd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:01.933099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:01.933735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:01.934007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:01.934313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] PATH_INFO: `/v3/users/b08503068f3c4bfc848ad6208fe2f558` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:01.935458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08503068f3c4bfc848ad6208fe2f558)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:01.937499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:01.991153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08503068f3c4bfc848ad6208fe2f558'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:01.991805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08503068f3c4bfc848ad6208fe2f558'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.004506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08503068f3c4bfc848ad6208fe2f558'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.005646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:02.013266 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08503068f3c4bfc848ad6208fe2f558'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.013949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08503068f3c4bfc848ad6208fe2f558'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.030404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1018/2039] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:01 2022] DELETE /identity/v3/users/b08503068f3c4bfc848ad6208fe2f558 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:02.035230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0c4e-be14-492e-8019-04c4739dc3e5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.035536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0c4e-be14-492e-8019-04c4739dc3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.035808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0c4e-be14-492e-8019-04c4739dc3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.050414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:02.056768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-921b0c4e-be14-492e-8019-04c4739dc3e5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:02.058772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-921b0c4e-be14-492e-8019-04c4739dc3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:02.119207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1022/2040] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:02.131026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.132579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:02.197920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65f787-c18d-48b2-867e-3a2fb383fb88 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.202046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685dd1-8539-4d62-899c-13272e8b17df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.202688 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685dd1-8539-4d62-899c-13272e8b17df service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.203407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685dd1-8539-4d62-899c-13272e8b17df service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.203711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685dd1-8539-4d62-899c-13272e8b17df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.214677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b685dd1-8539-4d62-899c-13272e8b17df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:02.216122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b685dd1-8539-4d62-899c-13272e8b17df service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:02.227556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1019/2041] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:02.511229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b0c4e-be14-492e-8019-04c4739dc3e5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.512647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-921b0c4e-be14-492e-8019-04c4739dc3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:02.525669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b0c4e-be14-492e-8019-04c4739dc3e5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.530420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.531339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.532057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.532329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] PATH_INFO: `/v3/projects/892b1494626a45179257cbac98e5be0f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.533492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=892b1494626a45179257cbac98e5be0f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:02.535430 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:02.583518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892b1494626a45179257cbac98e5be0f', 'request_id': 'req-3706988d-4c99-4c72-8273-9e198f0bfe48'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.591843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892b1494626a45179257cbac98e5be0f', 'request_id': 'req-3706988d-4c99-4c72-8273-9e198f0bfe48'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.599077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892b1494626a45179257cbac98e5be0f', 'request_id': 'req-3706988d-4c99-4c72-8273-9e198f0bfe48'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.610345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] The token cache is being invalidate because project 892b1494626a45179257cbac98e5be0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:02.610634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.611302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.612870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1023/2042] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:02 2022] DELETE /identity/v3/projects/892b1494626a45179257cbac98e5be0f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:02.616929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685dd1-8539-4d62-899c-13272e8b17df service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.627079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685dd1-8539-4d62-899c-13272e8b17df service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.686843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.687511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.687808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.688096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] PATH_INFO: `/v3/users/7520c2c368be42bfab928d391b229f52` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.689447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7520c2c368be42bfab928d391b229f52)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:02.691775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:02.746258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7520c2c368be42bfab928d391b229f52'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.746876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7520c2c368be42bfab928d391b229f52'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.760196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7520c2c368be42bfab928d391b229f52'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.767427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7520c2c368be42bfab928d391b229f52'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.768136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79e72a1b-df6e-4596-a3c2-6de752edcc99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7520c2c368be42bfab928d391b229f52'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:02.786463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1020/2043] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:02 2022] DELETE /identity/v3/users/7520c2c368be42bfab928d391b229f52 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:03.076970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.082145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:03.094901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706988d-4c99-4c72-8273-9e198f0bfe48 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.158291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.158907 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.159182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.159453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] PATH_INFO: `/v3/projects/988cab6db7d64312a95b63bcf036b430` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.160620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=988cab6db7d64312a95b63bcf036b430)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.162543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.200128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:03.215407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988cab6db7d64312a95b63bcf036b430', 'request_id': 'req-46a07395-7442-4731-a3e6-e62cfa0a5f60'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.225137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988cab6db7d64312a95b63bcf036b430', 'request_id': 'req-46a07395-7442-4731-a3e6-e62cfa0a5f60'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.231730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988cab6db7d64312a95b63bcf036b430', 'request_id': 'req-46a07395-7442-4731-a3e6-e62cfa0a5f60'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.242886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] The token cache is being invalidate because project 988cab6db7d64312a95b63bcf036b430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:03.243234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.243879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-46a07395-7442-4731-a3e6-e62cfa0a5f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.245448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1024/2044] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:03 2022] DELETE /identity/v3/projects/988cab6db7d64312a95b63bcf036b430 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:03.250630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae398f4-0be2-47ed-96cf-6da1b9b820ad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.251139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae398f4-0be2-47ed-96cf-6da1b9b820ad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.251139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae398f4-0be2-47ed-96cf-6da1b9b820ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.259917 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:03.272441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fae398f4-0be2-47ed-96cf-6da1b9b820ad None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:03.274241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fae398f4-0be2-47ed-96cf-6da1b9b820ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:03.338942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1021/2045] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:03.352403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a948-59a8-4aa2-bdb8-8f432eeec284 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.352690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a948-59a8-4aa2-bdb8-8f432eeec284 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.352983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a948-59a8-4aa2-bdb8-8f432eeec284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.374468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b909a948-59a8-4aa2-bdb8-8f432eeec284 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:03.375672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b909a948-59a8-4aa2-bdb8-8f432eeec284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:03.385044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1025/2046] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:03.389442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae398f4-0be2-47ed-96cf-6da1b9b820ad None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.399933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae398f4-0be2-47ed-96cf-6da1b9b820ad None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.405862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.406863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.407141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.407408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.408051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:03.408359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.409957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.423017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1022/2047] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:03.427618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909a948-59a8-4aa2-bdb8-8f432eeec284 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.438349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909a948-59a8-4aa2-bdb8-8f432eeec284 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.443683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df997395-b6a6-4af3-9ed8-a595796c356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.444365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df997395-b6a6-4af3-9ed8-a595796c356e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.444640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df997395-b6a6-4af3-9ed8-a595796c356e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.444911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df997395-b6a6-4af3-9ed8-a595796c356e admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.445532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df997395-b6a6-4af3-9ed8-a595796c356e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.447458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df997395-b6a6-4af3-9ed8-a595796c356e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.482878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1026/2048] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:03 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:03.484549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.496187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80000c2-3ef3-482f-8f65-7473131b7ddf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.555545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fab327-6b2c-4fa5-8194-2bc68e998266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.556153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fab327-6b2c-4fa5-8194-2bc68e998266 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.556420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fab327-6b2c-4fa5-8194-2bc68e998266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.556691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fab327-6b2c-4fa5-8194-2bc68e998266 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.557313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fab327-6b2c-4fa5-8194-2bc68e998266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.559259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fab327-6b2c-4fa5-8194-2bc68e998266 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.582592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1023/2049] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:03 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:03.586318 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df997395-b6a6-4af3-9ed8-a595796c356e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.600056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df997395-b6a6-4af3-9ed8-a595796c356e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.606584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfba40-ab31-49b8-ba12-f1685dc68567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.607188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfba40-ab31-49b8-ba12-f1685dc68567 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.607466 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfba40-ab31-49b8-ba12-f1685dc68567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.607740 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfba40-ab31-49b8-ba12-f1685dc68567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.608381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfba40-ab31-49b8-ba12-f1685dc68567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.610509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfba40-ab31-49b8-ba12-f1685dc68567 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.621571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1027/2050] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:36:03.626165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fab327-6b2c-4fa5-8194-2bc68e998266 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.636773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fab327-6b2c-4fa5-8194-2bc68e998266 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.642830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57146503-c391-4dfa-8175-40aba45fda9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.643463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57146503-c391-4dfa-8175-40aba45fda9d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.643744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57146503-c391-4dfa-8175-40aba45fda9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.644012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57146503-c391-4dfa-8175-40aba45fda9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.644659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57146503-c391-4dfa-8175-40aba45fda9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.646280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57146503-c391-4dfa-8175-40aba45fda9d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.658251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1024/2051] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:03.662628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfba40-ab31-49b8-ba12-f1685dc68567 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.673495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfba40-ab31-49b8-ba12-f1685dc68567 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.678389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744aedc-bb42-460e-bc37-5dbd113720aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.679033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744aedc-bb42-460e-bc37-5dbd113720aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.679308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744aedc-bb42-460e-bc37-5dbd113720aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.679577 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744aedc-bb42-460e-bc37-5dbd113720aa admin admin] PATH_INFO: `/v3/projects/9f280bbd3f564fe1a7a544b17d25d1db/users/49cf148f11dd40fd9be426e8bf640383/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.695518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744aedc-bb42-460e-bc37-5dbd113720aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f280bbd3f564fe1a7a544b17d25d1db, user_id=49cf148f11dd40fd9be426e8bf640383, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.698262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744aedc-bb42-460e-bc37-5dbd113720aa admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.711975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1028/2052] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:03 2022] PUT /identity/v3/projects/9f280bbd3f564fe1a7a544b17d25d1db/users/49cf148f11dd40fd9be426e8bf640383/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:03.723074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e9d6-62c6-423b-9533-4ea85f444c7b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.723350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e9d6-62c6-423b-9533-4ea85f444c7b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.723630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e9d6-62c6-423b-9533-4ea85f444c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.752495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2ad7e9d6-62c6-423b-9533-4ea85f444c7b None None] MFA Rules not processed for user `49cf148f11dd40fd9be426e8bf640383`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:03.753639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2ad7e9d6-62c6-423b-9533-4ea85f444c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:03.824762 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:03.828855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1025/2053] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:03.834403 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744aedc-bb42-460e-bc37-5dbd113720aa admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.906649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744aedc-bb42-460e-bc37-5dbd113720aa admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.912401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51264521-9f6d-48d1-915b-61ed69b0a0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.913004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51264521-9f6d-48d1-915b-61ed69b0a0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.913272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51264521-9f6d-48d1-915b-61ed69b0a0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.913539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51264521-9f6d-48d1-915b-61ed69b0a0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.914217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51264521-9f6d-48d1-915b-61ed69b0a0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.916108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51264521-9f6d-48d1-915b-61ed69b0a0f8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.947948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1029/2054] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:03 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:03.952223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7e9d6-62c6-423b-9533-4ea85f444c7b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.963988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7e9d6-62c6-423b-9533-4ea85f444c7b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.028312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c913157-a865-427c-9484-1e9613c497a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.028825 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913157-a865-427c-9484-1e9613c497a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.029095 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913157-a865-427c-9484-1e9613c497a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.029360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913157-a865-427c-9484-1e9613c497a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.029979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c913157-a865-427c-9484-1e9613c497a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.031933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c913157-a865-427c-9484-1e9613c497a7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.055890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1026/2055] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:03 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:04.059815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51264521-9f6d-48d1-915b-61ed69b0a0f8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:04.070278 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51264521-9f6d-48d1-915b-61ed69b0a0f8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.075144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd5638-2f46-4dd6-9348-f73389262f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.075763 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd5638-2f46-4dd6-9348-f73389262f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.076044 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd5638-2f46-4dd6-9348-f73389262f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.076314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd5638-2f46-4dd6-9348-f73389262f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.076948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd5638-2f46-4dd6-9348-f73389262f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.078988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd5638-2f46-4dd6-9348-f73389262f06 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.089657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1030/2056] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:04.094245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c913157-a865-427c-9484-1e9613c497a7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:04.104973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c913157-a865-427c-9484-1e9613c497a7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.109907 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619d2b2-d4b5-4bb8-bb0a-0f7f88a17994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.110591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d2b2-d4b5-4bb8-bb0a-0f7f88a17994 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.110861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d2b2-d4b5-4bb8-bb0a-0f7f88a17994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.111158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d2b2-d4b5-4bb8-bb0a-0f7f88a17994 admin admin] PATH_INFO: `/v3/projects/ba5e79e821db4415be56ee64104512ce/users/cec37d513f6d4ccda5ab8112576a6692/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.128589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619d2b2-d4b5-4bb8-bb0a-0f7f88a17994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5e79e821db4415be56ee64104512ce, user_id=cec37d513f6d4ccda5ab8112576a6692, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.131533 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619d2b2-d4b5-4bb8-bb0a-0f7f88a17994 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.147043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1027/2057] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:04 2022] PUT /identity/v3/projects/ba5e79e821db4415be56ee64104512ce/users/cec37d513f6d4ccda5ab8112576a6692/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:04.151662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd5638-2f46-4dd6-9348-f73389262f06 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:04.162923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd5638-2f46-4dd6-9348-f73389262f06 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.192660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:04.224078 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb2cdf-db33-4cd0-86eb-4c88aa88a533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.224700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2cdf-db33-4cd0-86eb-4c88aa88a533 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.224968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2cdf-db33-4cd0-86eb-4c88aa88a533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.225231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2cdf-db33-4cd0-86eb-4c88aa88a533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.225873 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb2cdf-db33-4cd0-86eb-4c88aa88a533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.227526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb2cdf-db33-4cd0-86eb-4c88aa88a533 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.238791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1031/2058] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:04.242924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619d2b2-d4b5-4bb8-bb0a-0f7f88a17994 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:04.253585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619d2b2-d4b5-4bb8-bb0a-0f7f88a17994 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.258519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.259149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.259419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.259681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] PATH_INFO: `/v3/projects/ba5e79e821db4415be56ee64104512ce/users/cec37d513f6d4ccda5ab8112576a6692/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.261648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5e79e821db4415be56ee64104512ce, user_id=cec37d513f6d4ccda5ab8112576a6692, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.264267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.274171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1028/2059] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:04 2022] PUT /identity/v3/projects/ba5e79e821db4415be56ee64104512ce/users/cec37d513f6d4ccda5ab8112576a6692/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:04.284686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ab9d6-0386-4ed2-8d0c-05dcd1348775 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.284878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ab9d6-0386-4ed2-8d0c-05dcd1348775 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.285203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ab9d6-0386-4ed2-8d0c-05dcd1348775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.318902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2c9ab9d6-0386-4ed2-8d0c-05dcd1348775 None None] MFA Rules not processed for user `cec37d513f6d4ccda5ab8112576a6692`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:04.320156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c9ab9d6-0386-4ed2-8d0c-05dcd1348775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:04.394533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1032/2060] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:04.406227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:04.408107 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:04.470355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b80c26-938c-4f39-9075-a7f3c21af56a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.475473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.476070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.476364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.476625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.489852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.492092 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.504788 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1029/2061] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:04.594785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ab9d6-0386-4ed2-8d0c-05dcd1348775 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:04.595976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c9ab9d6-0386-4ed2-8d0c-05dcd1348775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:04.608893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ab9d6-0386-4ed2-8d0c-05dcd1348775 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.613013 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bc8c2-cfb1-40bd-8212-4434e6d17e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.613618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc8c2-cfb1-40bd-8212-4434e6d17e90 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.613892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc8c2-cfb1-40bd-8212-4434e6d17e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.614193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc8c2-cfb1-40bd-8212-4434e6d17e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.674833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bc8c2-cfb1-40bd-8212-4434e6d17e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.676303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bc8c2-cfb1-40bd-8212-4434e6d17e90 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.685124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1033/2062] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:05.023622 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:05.025636 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:05.085549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:05.102825 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec9b03-cd0b-43fe-a876-5385ffd79e0b service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:05.108097 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:05.108727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:05.108996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:05.109260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:05.111310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:05.124212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:05.125937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:05.136191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1030/2063] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:13.295112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134915-32a5-4d36-862b-b80a8720d7f9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:13.295609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134915-32a5-4d36-862b-b80a8720d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:13.295715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134915-32a5-4d36-862b-b80a8720d7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:13.316722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0b134915-32a5-4d36-862b-b80a8720d7f9 None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:13.317942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0b134915-32a5-4d36-862b-b80a8720d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:13.366718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:13.377986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1034/2064] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Nov 2 07:36:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:13.390486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:13.392096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:13.467656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e772943-a44f-4e12-a822-9c46277da730 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:13.473754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74c27-0a65-4220-8024-a8c3299d868b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:13.474389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74c27-0a65-4220-8024-a8c3299d868b service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:13.474670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74c27-0a65-4220-8024-a8c3299d868b service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:13.474940 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74c27-0a65-4220-8024-a8c3299d868b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:13.483292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74c27-0a65-4220-8024-a8c3299d868b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:13.485090 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74c27-0a65-4220-8024-a8c3299d868b service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:13.495369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1031/2065] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:36:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:36:23.372028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:23.491399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=39 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:39.512955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19508-5198-4e6f-9d86-6a44b967dc5d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.514031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19508-5198-4e6f-9d86-6a44b967dc5d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.514364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19508-5198-4e6f-9d86-6a44b967dc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.536015 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3af19508-5198-4e6f-9d86-6a44b967dc5d None None] MFA Rules not processed for user `55fd5764905e4dfa899a90270181b220`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:39.537198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3af19508-5198-4e6f-9d86-6a44b967dc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:39.545201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1035/2066] 10.222.0.25 () {58 vars in 967 bytes} [Wed Nov 2 07:36:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:39.568469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74c27-0a65-4220-8024-a8c3299d868b service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:39.580794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74c27-0a65-4220-8024-a8c3299d868b service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:39.586994 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6578f6-07dd-40ca-83b6-f46235c1c7ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab55070be14274857478b5581d2a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:39.587587 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6578f6-07dd-40ca-83b6-f46235c1c7ee service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:39.587864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6578f6-07dd-40ca-83b6-f46235c1c7ee service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:39.588130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6578f6-07dd-40ca-83b6-f46235c1c7ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:39.598270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6578f6-07dd-40ca-83b6-f46235c1c7ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:39.599859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6578f6-07dd-40ca-83b6-f46235c1c7ee service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:39.608790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1032/2067] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Nov 2 07:36:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:36:49.530246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:49.605183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:51.222511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19508-5198-4e6f-9d86-6a44b967dc5d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:51.293281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19508-5198-4e6f-9d86-6a44b967dc5d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:51.301329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:51.302052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:51.302442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:51.302816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] PATH_INFO: `/v3/users/49cf148f11dd40fd9be426e8bf640383` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:51.304791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=49cf148f11dd40fd9be426e8bf640383)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:51.307847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:51.363170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49cf148f11dd40fd9be426e8bf640383'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.363788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49cf148f11dd40fd9be426e8bf640383'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.376196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49cf148f11dd40fd9be426e8bf640383'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.383999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49cf148f11dd40fd9be426e8bf640383'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.384751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49cf148f11dd40fd9be426e8bf640383'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.400143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1036/2068] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:51 2022] DELETE /identity/v3/users/49cf148f11dd40fd9be426e8bf640383 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:51.404982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd9c8c-a4ec-4798-b2ca-8184a76e074e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:51.405501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd9c8c-a4ec-4798-b2ca-8184a76e074e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:51.405614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd9c8c-a4ec-4798-b2ca-8184a76e074e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:51.425707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b1dd9c8c-a4ec-4798-b2ca-8184a76e074e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:51.426943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b1dd9c8c-a4ec-4798-b2ca-8184a76e074e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:51.499020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1033/2069] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:36:51.510571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:51.512042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:51.595209 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b09f7e-a224-43bb-b7e8-46e096939c5c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:51.599699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ba56c-be21-4f33-b621-ca9705320f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:51.600316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba56c-be21-4f33-b621-ca9705320f02 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:51.600593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba56c-be21-4f33-b621-ca9705320f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:51.600865 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba56c-be21-4f33-b621-ca9705320f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:51.610432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ba56c-be21-4f33-b621-ca9705320f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:51.611875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ba56c-be21-4f33-b621-ca9705320f02 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:51.621914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1037/2070] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:51.900519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd9c8c-a4ec-4798-b2ca-8184a76e074e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:51.902025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b1dd9c8c-a4ec-4798-b2ca-8184a76e074e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:51.925260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd9c8c-a4ec-4798-b2ca-8184a76e074e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:51.930542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:51.931211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:51.931506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:51.931773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] PATH_INFO: `/v3/projects/9f280bbd3f564fe1a7a544b17d25d1db` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:51.932884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f280bbd3f564fe1a7a544b17d25d1db)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:51.934874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:51.985038 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f280bbd3f564fe1a7a544b17d25d1db', 'request_id': 'req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:51.992417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f280bbd3f564fe1a7a544b17d25d1db', 'request_id': 'req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.002647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f280bbd3f564fe1a7a544b17d25d1db', 'request_id': 'req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.014766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] The token cache is being invalidate because project 9f280bbd3f564fe1a7a544b17d25d1db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:52.015049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.015869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.017786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1034/2071] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:51 2022] DELETE /identity/v3/projects/9f280bbd3f564fe1a7a544b17d25d1db => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:52.021658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ba56c-be21-4f33-b621-ca9705320f02 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.026563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:52.033387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ba56c-be21-4f33-b621-ca9705320f02 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.095382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.095998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.096615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.096881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] PATH_INFO: `/v3/users/cec37d513f6d4ccda5ab8112576a6692` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.098077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec37d513f6d4ccda5ab8112576a6692)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.100015 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.148331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec37d513f6d4ccda5ab8112576a6692'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.148890 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec37d513f6d4ccda5ab8112576a6692'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.161560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec37d513f6d4ccda5ab8112576a6692'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.168938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec37d513f6d4ccda5ab8112576a6692'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.169683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30aa34ad-9036-4f4c-8307-40567fbd4cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec37d513f6d4ccda5ab8112576a6692'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.188277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1038/2072] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:52 2022] DELETE /identity/v3/users/cec37d513f6d4ccda5ab8112576a6692 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:52.471596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.473235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:52.486980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d627d-abaf-4dfa-8c1f-80653a2ac357 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.546655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.547281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.547556 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.547824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] PATH_INFO: `/v3/projects/ba5e79e821db4415be56ee64104512ce` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.549019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba5e79e821db4415be56ee64104512ce)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.550955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.585475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:52.604593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba5e79e821db4415be56ee64104512ce', 'request_id': 'req-b9466e7e-79a9-409b-976b-da279cf1f3b3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.611060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba5e79e821db4415be56ee64104512ce', 'request_id': 'req-b9466e7e-79a9-409b-976b-da279cf1f3b3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.620895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba5e79e821db4415be56ee64104512ce', 'request_id': 'req-b9466e7e-79a9-409b-976b-da279cf1f3b3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.631619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] The token cache is being invalidate because project ba5e79e821db4415be56ee64104512ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:52.631900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.632567 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9466e7e-79a9-409b-976b-da279cf1f3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:52.634096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1035/2073] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:52 2022] DELETE /identity/v3/projects/ba5e79e821db4415be56ee64104512ce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:52.639231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef7ce-3e5d-4bd1-bae0-3831312e7e71 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.639749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef7ce-3e5d-4bd1-bae0-3831312e7e71 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.639843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef7ce-3e5d-4bd1-bae0-3831312e7e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.661180 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4ceef7ce-3e5d-4bd1-bae0-3831312e7e71 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:52.662351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4ceef7ce-3e5d-4bd1-bae0-3831312e7e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:52.727269 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1039/2074] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:52.735607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475c823-1c59-4696-8466-84b2ce619a1c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.735874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475c823-1c59-4696-8466-84b2ce619a1c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.736161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475c823-1c59-4696-8466-84b2ce619a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.756621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9475c823-1c59-4696-8466-84b2ce619a1c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:52.757818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9475c823-1c59-4696-8466-84b2ce619a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:52.765828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1036/2075] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:52.770169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceef7ce-3e5d-4bd1-bae0-3831312e7e71 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.781189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceef7ce-3e5d-4bd1-bae0-3831312e7e71 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.787024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.787024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.787024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.787024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.788606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:52.788606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.788606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.801112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1040/2076] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:52.805639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475c823-1c59-4696-8466-84b2ce619a1c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.816127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475c823-1c59-4696-8466-84b2ce619a1c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.821152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ccf3-4ac4-4551-a16a-5f2affdc818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.821747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ccf3-4ac4-4551-a16a-5f2affdc818c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.822134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ccf3-4ac4-4551-a16a-5f2affdc818c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.822317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ccf3-4ac4-4551-a16a-5f2affdc818c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.823235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355ccf3-4ac4-4551-a16a-5f2affdc818c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.825500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355ccf3-4ac4-4551-a16a-5f2affdc818c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.858687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1037/2077] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:52.863739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.873919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3807f0-8737-4fcf-a739-fc64f4a50941 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.879144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:52.938004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aee41-9853-415c-96a5-01a39ee6bb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.939261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aee41-9853-415c-96a5-01a39ee6bb26 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.939542 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aee41-9853-415c-96a5-01a39ee6bb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.939815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aee41-9853-415c-96a5-01a39ee6bb26 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.940452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aee41-9853-415c-96a5-01a39ee6bb26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.942418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aee41-9853-415c-96a5-01a39ee6bb26 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.965299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1041/2078] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:52 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:52.969680 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ccf3-4ac4-4551-a16a-5f2affdc818c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:52.980464 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ccf3-4ac4-4551-a16a-5f2affdc818c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:52.986059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae219e72-cd7f-4ab1-b68f-ce48b4f07172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:52.986674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219e72-cd7f-4ab1-b68f-ce48b4f07172 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:52.986947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219e72-cd7f-4ab1-b68f-ce48b4f07172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:52.987211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219e72-cd7f-4ab1-b68f-ce48b4f07172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:52.987852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae219e72-cd7f-4ab1-b68f-ce48b4f07172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:52.989451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae219e72-cd7f-4ab1-b68f-ce48b4f07172 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:52.999869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1038/2079] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:53.004170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aee41-9853-415c-96a5-01a39ee6bb26 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.014796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aee41-9853-415c-96a5-01a39ee6bb26 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.020628 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c22e4-dffa-4a1f-9f8c-1af373b65455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.021277 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c22e4-dffa-4a1f-9f8c-1af373b65455 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.021555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c22e4-dffa-4a1f-9f8c-1af373b65455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.021825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c22e4-dffa-4a1f-9f8c-1af373b65455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.022486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c22e4-dffa-4a1f-9f8c-1af373b65455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.024088 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c22e4-dffa-4a1f-9f8c-1af373b65455 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.035113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1042/2080] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:53.039315 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae219e72-cd7f-4ab1-b68f-ce48b4f07172 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.049727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae219e72-cd7f-4ab1-b68f-ce48b4f07172 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.055679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30229aa-9391-405a-b345-56620db6eb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.056348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30229aa-9391-405a-b345-56620db6eb09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.056618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30229aa-9391-405a-b345-56620db6eb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.056871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30229aa-9391-405a-b345-56620db6eb09 admin admin] PATH_INFO: `/v3/projects/772c0a222fe5486595262650b149aacb/users/af28eae1df67402bbdcdf31353e87bad/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.073417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30229aa-9391-405a-b345-56620db6eb09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772c0a222fe5486595262650b149aacb, user_id=af28eae1df67402bbdcdf31353e87bad, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.076074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30229aa-9391-405a-b345-56620db6eb09 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.092718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1039/2081] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:53 2022] PUT /identity/v3/projects/772c0a222fe5486595262650b149aacb/users/af28eae1df67402bbdcdf31353e87bad/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:53.166721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df523f1b-250c-4bbe-bbea-6b41345958cb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.166962 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df523f1b-250c-4bbe-bbea-6b41345958cb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.167175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df523f1b-250c-4bbe-bbea-6b41345958cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.196107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-df523f1b-250c-4bbe-bbea-6b41345958cb None None] MFA Rules not processed for user `af28eae1df67402bbdcdf31353e87bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:53.197337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-df523f1b-250c-4bbe-bbea-6b41345958cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:53.262990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1043/2082] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:53.268082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30229aa-9391-405a-b345-56620db6eb09 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.340183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30229aa-9391-405a-b345-56620db6eb09 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.345982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf27d15-307b-48ad-9202-d2bf30bffaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.346961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27d15-307b-48ad-9202-d2bf30bffaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.347234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27d15-307b-48ad-9202-d2bf30bffaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.347497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf27d15-307b-48ad-9202-d2bf30bffaa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.348122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf27d15-307b-48ad-9202-d2bf30bffaa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.350020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf27d15-307b-48ad-9202-d2bf30bffaa5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.381273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1040/2083] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:53 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:53.385051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df523f1b-250c-4bbe-bbea-6b41345958cb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.396192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df523f1b-250c-4bbe-bbea-6b41345958cb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.461909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41065545-c09f-4378-8ce9-fa240c9d08a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.462453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41065545-c09f-4378-8ce9-fa240c9d08a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.462724 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41065545-c09f-4378-8ce9-fa240c9d08a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.462990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41065545-c09f-4378-8ce9-fa240c9d08a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.463619 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41065545-c09f-4378-8ce9-fa240c9d08a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.465533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41065545-c09f-4378-8ce9-fa240c9d08a9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.492271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1044/2084] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:53 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:53.496821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf27d15-307b-48ad-9202-d2bf30bffaa5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.508820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf27d15-307b-48ad-9202-d2bf30bffaa5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.513408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ec7fe-2e0b-4df5-9d48-75ae7727075d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.514004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec7fe-2e0b-4df5-9d48-75ae7727075d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.514306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec7fe-2e0b-4df5-9d48-75ae7727075d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.514573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec7fe-2e0b-4df5-9d48-75ae7727075d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.515212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ec7fe-2e0b-4df5-9d48-75ae7727075d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.516888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ec7fe-2e0b-4df5-9d48-75ae7727075d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.532153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1041/2085] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:53.536588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41065545-c09f-4378-8ce9-fa240c9d08a9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.546708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41065545-c09f-4378-8ce9-fa240c9d08a9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.552095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82319fd-e5e7-42d5-a415-89aac880f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.552738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82319fd-e5e7-42d5-a415-89aac880f953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.553012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82319fd-e5e7-42d5-a415-89aac880f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.553278 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82319fd-e5e7-42d5-a415-89aac880f953 admin admin] PATH_INFO: `/v3/projects/41b5240d19944d44a0c218cec18570a2/users/ce9a1c2b45174d4bb6e7e38893458081/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.555527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:53.569913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82319fd-e5e7-42d5-a415-89aac880f953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b5240d19944d44a0c218cec18570a2, user_id=ce9a1c2b45174d4bb6e7e38893458081, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.572581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82319fd-e5e7-42d5-a415-89aac880f953 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.586436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1045/2086] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:53 2022] PUT /identity/v3/projects/41b5240d19944d44a0c218cec18570a2/users/ce9a1c2b45174d4bb6e7e38893458081/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:53.590470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ec7fe-2e0b-4df5-9d48-75ae7727075d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.601410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ec7fe-2e0b-4df5-9d48-75ae7727075d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.622896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:53.664167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1de2-5837-4712-9e8a-62b887ad47fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.664767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1de2-5837-4712-9e8a-62b887ad47fd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.665040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1de2-5837-4712-9e8a-62b887ad47fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.665307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1de2-5837-4712-9e8a-62b887ad47fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.665935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f1de2-5837-4712-9e8a-62b887ad47fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.667573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f1de2-5837-4712-9e8a-62b887ad47fd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.678212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1042/2087] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:53.684344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82319fd-e5e7-42d5-a415-89aac880f953 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.697862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82319fd-e5e7-42d5-a415-89aac880f953 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.706505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.707160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.707439 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.707712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] PATH_INFO: `/v3/projects/41b5240d19944d44a0c218cec18570a2/users/ce9a1c2b45174d4bb6e7e38893458081/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.709827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b5240d19944d44a0c218cec18570a2, user_id=ce9a1c2b45174d4bb6e7e38893458081, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.712678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.724749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1046/2088] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:53 2022] PUT /identity/v3/projects/41b5240d19944d44a0c218cec18570a2/users/ce9a1c2b45174d4bb6e7e38893458081/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:53.735122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d6447-cfec-4337-ae8d-225f7776660d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.735675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d6447-cfec-4337-ae8d-225f7776660d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.736023 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d6447-cfec-4337-ae8d-225f7776660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.764347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8b7d6447-cfec-4337-ae8d-225f7776660d None None] MFA Rules not processed for user `ce9a1c2b45174d4bb6e7e38893458081`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:53.765545 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8b7d6447-cfec-4337-ae8d-225f7776660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:53.842572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1043/2089] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:53.854890 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:53.856920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:53.955187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fffe4-2892-4377-a26a-101f58655a87 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:53.958991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c8fb5-4bde-4cd4-9fc1-e4f7e1bcd8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:53.959594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c8fb5-4bde-4cd4-9fc1-e4f7e1bcd8fc service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:53.959869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c8fb5-4bde-4cd4-9fc1-e4f7e1bcd8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:53.960140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c8fb5-4bde-4cd4-9fc1-e4f7e1bcd8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:53.973364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c8fb5-4bde-4cd4-9fc1-e4f7e1bcd8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:53.974827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c8fb5-4bde-4cd4-9fc1-e4f7e1bcd8fc service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:53.984708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1047/2090] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:54.017642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d6447-cfec-4337-ae8d-225f7776660d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:54.019277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8b7d6447-cfec-4337-ae8d-225f7776660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:54.065964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c8fb5-4bde-4cd4-9fc1-e4f7e1bcd8fc service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:54.074581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c8fb5-4bde-4cd4-9fc1-e4f7e1bcd8fc service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:54.078175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663b5e9-2354-4302-8ebc-876c0d7c59c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:54.078766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663b5e9-2354-4302-8ebc-876c0d7c59c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:54.078919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d6447-cfec-4337-ae8d-225f7776660d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:54.079086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663b5e9-2354-4302-8ebc-876c0d7c59c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:54.079883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663b5e9-2354-4302-8ebc-876c0d7c59c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:54.082453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4f8e2-08a6-462a-a52c-d743620c9425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:54.083059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f8e2-08a6-462a-a52c-d743620c9425 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:54.083336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f8e2-08a6-462a-a52c-d743620c9425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:54.083605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4f8e2-08a6-462a-a52c-d743620c9425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:54.085302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-10b4f8e2-08a6-462a-a52c-d743620c9425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:54.139467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663b5e9-2354-4302-8ebc-876c0d7c59c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:54.140889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663b5e9-2354-4302-8ebc-876c0d7c59c8 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:54.149902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1048/2091] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:54.152038 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:54.153565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4f8e2-08a6-462a-a52c-d743620c9425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:54.155330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4f8e2-08a6-462a-a52c-d743620c9425 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:54.166476 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1044/2092] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:54.427504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663b5e9-2354-4302-8ebc-876c0d7c59c8 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:54.439052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663b5e9-2354-4302-8ebc-876c0d7c59c8 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:54.443573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:54.444219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:54.444553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:54.444829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] PATH_INFO: `/v3/users/af28eae1df67402bbdcdf31353e87bad` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:54.445995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af28eae1df67402bbdcdf31353e87bad)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:54.447995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:54.503080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:54.504914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af28eae1df67402bbdcdf31353e87bad'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.505849 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af28eae1df67402bbdcdf31353e87bad'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.521662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af28eae1df67402bbdcdf31353e87bad'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.529328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af28eae1df67402bbdcdf31353e87bad'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.530107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af28eae1df67402bbdcdf31353e87bad'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:54.547436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1049/2093] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:54 2022] DELETE /identity/v3/users/af28eae1df67402bbdcdf31353e87bad => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:54.552915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee39fab-bf06-4f9d-9299-4057ff8d9ff7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:54.553473 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee39fab-bf06-4f9d-9299-4057ff8d9ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:54.553579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee39fab-bf06-4f9d-9299-4057ff8d9ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:54.574744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3ee39fab-bf06-4f9d-9299-4057ff8d9ff7 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:54.575987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ee39fab-bf06-4f9d-9299-4057ff8d9ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:54.643542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1045/2094] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:54.653680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:54.655193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:54.715434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562e4a4-8fe4-4a10-a28e-bf4256ed2337 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:54.719183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abba315-0e55-422b-b424-152db52453b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:54.720065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abba315-0e55-422b-b424-152db52453b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:54.720539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abba315-0e55-422b-b424-152db52453b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:54.721974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abba315-0e55-422b-b424-152db52453b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:54.731944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abba315-0e55-422b-b424-152db52453b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:54.733378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abba315-0e55-422b-b424-152db52453b6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:54.743950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1050/2095] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:55.024666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee39fab-bf06-4f9d-9299-4057ff8d9ff7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.027471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ee39fab-bf06-4f9d-9299-4057ff8d9ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.044323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee39fab-bf06-4f9d-9299-4057ff8d9ff7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.051180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.051826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.052140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.052377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] PATH_INFO: `/v3/projects/772c0a222fe5486595262650b149aacb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.053742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=772c0a222fe5486595262650b149aacb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.055911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.106916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772c0a222fe5486595262650b149aacb', 'request_id': 'req-f96de969-daf2-4824-b4c4-c360c06ce590'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.113393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772c0a222fe5486595262650b149aacb', 'request_id': 'req-f96de969-daf2-4824-b4c4-c360c06ce590'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.123256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772c0a222fe5486595262650b149aacb', 'request_id': 'req-f96de969-daf2-4824-b4c4-c360c06ce590'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.134439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] The token cache is being invalidate because project 772c0a222fe5486595262650b149aacb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:55.134720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.135347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.136837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1046/2096] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:55 2022] DELETE /identity/v3/projects/772c0a222fe5486595262650b149aacb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:55.140878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abba315-0e55-422b-b424-152db52453b6 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.150836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abba315-0e55-422b-b424-152db52453b6 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.207687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.208333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.208603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.208913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] PATH_INFO: `/v3/users/ce9a1c2b45174d4bb6e7e38893458081` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.210190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9a1c2b45174d4bb6e7e38893458081)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.212127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.261407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9a1c2b45174d4bb6e7e38893458081'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.262012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9a1c2b45174d4bb6e7e38893458081'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.274971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9a1c2b45174d4bb6e7e38893458081'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.277326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:55.284810 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9a1c2b45174d4bb6e7e38893458081'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.285782 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30ed70b8-bf15-4666-a402-02bd8cc8f152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9a1c2b45174d4bb6e7e38893458081'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.305803 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1051/2097] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:55 2022] DELETE /identity/v3/users/ce9a1c2b45174d4bb6e7e38893458081 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:55.619031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.620576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.646306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96de969-daf2-4824-b4c4-c360c06ce590 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.722787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.723538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.723872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.724196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] PATH_INFO: `/v3/projects/41b5240d19944d44a0c218cec18570a2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.725924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b5240d19944d44a0c218cec18570a2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.727897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:55.754736 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:55.784538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b5240d19944d44a0c218cec18570a2', 'request_id': 'req-e29126f0-0733-468b-ba7d-1a142b2ff475'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.791182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b5240d19944d44a0c218cec18570a2', 'request_id': 'req-e29126f0-0733-468b-ba7d-1a142b2ff475'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.800844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b5240d19944d44a0c218cec18570a2', 'request_id': 'req-e29126f0-0733-468b-ba7d-1a142b2ff475'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.813774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] The token cache is being invalidate because project 41b5240d19944d44a0c218cec18570a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:55.814052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.814765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e29126f0-0733-468b-ba7d-1a142b2ff475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:55.816256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1047/2098] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:55 2022] DELETE /identity/v3/projects/41b5240d19944d44a0c218cec18570a2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:55.821348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1213e88-b070-456a-bcf7-b2ff9fcd30db None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.821837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1213e88-b070-456a-bcf7-b2ff9fcd30db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.821940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1213e88-b070-456a-bcf7-b2ff9fcd30db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.843550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a1213e88-b070-456a-bcf7-b2ff9fcd30db None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:55.844834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a1213e88-b070-456a-bcf7-b2ff9fcd30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.924230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1052/2099] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:55.925984 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5955d-7b96-42af-8f84-1f71b0ec7ea2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.926501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5955d-7b96-42af-8f84-1f71b0ec7ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.926796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5955d-7b96-42af-8f84-1f71b0ec7ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.950905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-16d5955d-7b96-42af-8f84-1f71b0ec7ea2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:55.952683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-16d5955d-7b96-42af-8f84-1f71b0ec7ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:55.964411 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1048/2100] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:55.965835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1213e88-b070-456a-bcf7-b2ff9fcd30db None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:55.977349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1213e88-b070-456a-bcf7-b2ff9fcd30db None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:55.986233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:55.986928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:55.987208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:55.988000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:55.988879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:55.989185 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:55.991566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.010480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1053/2101] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:36:56.011864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5955d-7b96-42af-8f84-1f71b0ec7ea2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.026145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5955d-7b96-42af-8f84-1f71b0ec7ea2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.032269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04822208-6c15-41c8-b807-af5518c5e23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.035025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04822208-6c15-41c8-b807-af5518c5e23d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.035270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04822208-6c15-41c8-b807-af5518c5e23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.035546 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04822208-6c15-41c8-b807-af5518c5e23d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.036215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04822208-6c15-41c8-b807-af5518c5e23d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.038193 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04822208-6c15-41c8-b807-af5518c5e23d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.076440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1049/2102] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:56 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:56.080949 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.091874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f6563-5728-453e-beed-69d1104002f5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.175892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7f0e7-4d67-44f8-82ff-d34e1497467f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.176607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7f0e7-4d67-44f8-82ff-d34e1497467f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.176897 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7f0e7-4d67-44f8-82ff-d34e1497467f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.177174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7f0e7-4d67-44f8-82ff-d34e1497467f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.177894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7f0e7-4d67-44f8-82ff-d34e1497467f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.180021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7f0e7-4d67-44f8-82ff-d34e1497467f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.209166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1054/2103] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:56 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:56.213417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04822208-6c15-41c8-b807-af5518c5e23d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.226946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04822208-6c15-41c8-b807-af5518c5e23d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.233185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8665f83-7300-47b7-a22d-62b9fae762e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.234257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8665f83-7300-47b7-a22d-62b9fae762e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.234767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8665f83-7300-47b7-a22d-62b9fae762e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.235691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8665f83-7300-47b7-a22d-62b9fae762e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.236658 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8665f83-7300-47b7-a22d-62b9fae762e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.239408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8665f83-7300-47b7-a22d-62b9fae762e4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.251127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1050/2104] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:56.255242 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7f0e7-4d67-44f8-82ff-d34e1497467f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.267207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7f0e7-4d67-44f8-82ff-d34e1497467f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.273227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cf69f-3931-4358-88c7-2434c5f88b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.274581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf69f-3931-4358-88c7-2434c5f88b74 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.274581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf69f-3931-4358-88c7-2434c5f88b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.274876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf69f-3931-4358-88c7-2434c5f88b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.276018 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cf69f-3931-4358-88c7-2434c5f88b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.277643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cf69f-3931-4358-88c7-2434c5f88b74 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.293447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1055/2105] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:36:56.294614 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8665f83-7300-47b7-a22d-62b9fae762e4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.306076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8665f83-7300-47b7-a22d-62b9fae762e4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.312924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368d587-47c6-433e-970f-00c296df183a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.314509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368d587-47c6-433e-970f-00c296df183a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.314799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368d587-47c6-433e-970f-00c296df183a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.315522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368d587-47c6-433e-970f-00c296df183a admin admin] PATH_INFO: `/v3/projects/81efee57dccc4087aa35f43660dee692/users/1f7765a148a64712bcf65ad912b31aef/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.334181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368d587-47c6-433e-970f-00c296df183a admin admin] RBAC: Authorizing `identity:create_grant(project_id=81efee57dccc4087aa35f43660dee692, user_id=1f7765a148a64712bcf65ad912b31aef, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.336979 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368d587-47c6-433e-970f-00c296df183a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.363842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1051/2106] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:56 2022] PUT /identity/v3/projects/81efee57dccc4087aa35f43660dee692/users/1f7765a148a64712bcf65ad912b31aef/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:56.375088 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33093d51-4e89-4486-a82f-bc274b20a326 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.375366 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33093d51-4e89-4486-a82f-bc274b20a326 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.375645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33093d51-4e89-4486-a82f-bc274b20a326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.416629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-33093d51-4e89-4486-a82f-bc274b20a326 None None] MFA Rules not processed for user `1f7765a148a64712bcf65ad912b31aef`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:56.417975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-33093d51-4e89-4486-a82f-bc274b20a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:56.484003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:56.519134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1056/2107] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:56.519976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368d587-47c6-433e-970f-00c296df183a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.615212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368d587-47c6-433e-970f-00c296df183a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.622734 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2afaec-22c6-4b25-a0b9-84857e7b83d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.623387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2afaec-22c6-4b25-a0b9-84857e7b83d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.624420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2afaec-22c6-4b25-a0b9-84857e7b83d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.624634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2afaec-22c6-4b25-a0b9-84857e7b83d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.626182 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2afaec-22c6-4b25-a0b9-84857e7b83d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.629054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2afaec-22c6-4b25-a0b9-84857e7b83d3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.664847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1052/2108] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:56 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:56.666507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33093d51-4e89-4486-a82f-bc274b20a326 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.680075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33093d51-4e89-4486-a82f-bc274b20a326 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.753450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51be5e4-24d4-4004-97b6-312ca7f471f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.754073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51be5e4-24d4-4004-97b6-312ca7f471f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.754383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51be5e4-24d4-4004-97b6-312ca7f471f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.754657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51be5e4-24d4-4004-97b6-312ca7f471f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.755311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51be5e4-24d4-4004-97b6-312ca7f471f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.757246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51be5e4-24d4-4004-97b6-312ca7f471f1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.779789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1057/2109] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:56 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:56.784040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2afaec-22c6-4b25-a0b9-84857e7b83d3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.796552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2afaec-22c6-4b25-a0b9-84857e7b83d3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.805449 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c896fda-2721-4374-bd80-e873331552c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.806066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c896fda-2721-4374-bd80-e873331552c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.806426 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c896fda-2721-4374-bd80-e873331552c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.806693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c896fda-2721-4374-bd80-e873331552c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.807309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c896fda-2721-4374-bd80-e873331552c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.808916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c896fda-2721-4374-bd80-e873331552c1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.821041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1053/2110] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:56.825367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51be5e4-24d4-4004-97b6-312ca7f471f1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.837072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51be5e4-24d4-4004-97b6-312ca7f471f1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.842378 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e434cd-7032-42ac-b44d-dd37e67b9ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.843321 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e434cd-7032-42ac-b44d-dd37e67b9ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.843663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e434cd-7032-42ac-b44d-dd37e67b9ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.844001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e434cd-7032-42ac-b44d-dd37e67b9ab1 admin admin] PATH_INFO: `/v3/projects/701ac49cb9164da7abc11e823e175256/users/6d73a25467af4da4ae94fb26c0c278bb/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.863100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e434cd-7032-42ac-b44d-dd37e67b9ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=701ac49cb9164da7abc11e823e175256, user_id=6d73a25467af4da4ae94fb26c0c278bb, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.866261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e434cd-7032-42ac-b44d-dd37e67b9ab1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.887117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1058/2111] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:56 2022] PUT /identity/v3/projects/701ac49cb9164da7abc11e823e175256/users/6d73a25467af4da4ae94fb26c0c278bb/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:56.888308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c896fda-2721-4374-bd80-e873331552c1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.904719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c896fda-2721-4374-bd80-e873331552c1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:56.924464 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:56.968295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e9319-6390-40eb-849f-d551c2e48d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:56.969312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9319-6390-40eb-849f-d551c2e48d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:56.969660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9319-6390-40eb-849f-d551c2e48d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:56.969921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9319-6390-40eb-849f-d551c2e48d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:56.970621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e9319-6390-40eb-849f-d551c2e48d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:56.973082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e9319-6390-40eb-849f-d551c2e48d72 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:56.983919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1054/2112] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:36:56.988898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e434cd-7032-42ac-b44d-dd37e67b9ab1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:56.999984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e434cd-7032-42ac-b44d-dd37e67b9ab1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:57.005001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:57.005696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:57.005974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:57.007240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] PATH_INFO: `/v3/projects/701ac49cb9164da7abc11e823e175256/users/6d73a25467af4da4ae94fb26c0c278bb/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:57.009252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=701ac49cb9164da7abc11e823e175256, user_id=6d73a25467af4da4ae94fb26c0c278bb, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:57.013110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:57.027953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1059/2113] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:56 2022] PUT /identity/v3/projects/701ac49cb9164da7abc11e823e175256/users/6d73a25467af4da4ae94fb26c0c278bb/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:57.039029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043c142-d9cf-4e07-a838-4b222b121dbf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:57.039678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043c142-d9cf-4e07-a838-4b222b121dbf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:57.040175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043c142-d9cf-4e07-a838-4b222b121dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:57.077279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a043c142-d9cf-4e07-a838-4b222b121dbf None None] MFA Rules not processed for user `6d73a25467af4da4ae94fb26c0c278bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:57.078756 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a043c142-d9cf-4e07-a838-4b222b121dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:57.175453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1055/2114] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:57.186468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:57.188426 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:57.251284 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331630c-15e5-48fb-aaea-3d3e45beddbc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:57.254775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72278-4cde-44b7-9afb-ff1b5886929c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:57.255568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72278-4cde-44b7-9afb-ff1b5886929c service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:57.255817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72278-4cde-44b7-9afb-ff1b5886929c service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:57.256086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72278-4cde-44b7-9afb-ff1b5886929c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:57.306697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:57.320430 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72278-4cde-44b7-9afb-ff1b5886929c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:57.321888 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72278-4cde-44b7-9afb-ff1b5886929c service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:57.332003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1060/2115] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:57.715445 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043c142-d9cf-4e07-a838-4b222b121dbf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:57.717265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a043c142-d9cf-4e07-a838-4b222b121dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:57.797813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043c142-d9cf-4e07-a838-4b222b121dbf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:57.802772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:57.803401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:57.803675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:57.803982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] PATH_INFO: `/v3/users/8f151dc283e8498f961b49320130f770` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:57.805073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f151dc283e8498f961b49320130f770)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:57.807051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:57.854570 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:57.857368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f151dc283e8498f961b49320130f770'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:57.858160 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f151dc283e8498f961b49320130f770'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:57.872267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f151dc283e8498f961b49320130f770'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:57.879854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f151dc283e8498f961b49320130f770'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:57.880617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b5a29ee-af03-4d91-9aa8-b5f653aef5e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f151dc283e8498f961b49320130f770'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:57.894516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1056/2116] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:57 2022] DELETE /identity/v3/users/8f151dc283e8498f961b49320130f770 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:58.075888 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72278-4cde-44b7-9afb-ff1b5886929c service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.089137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72278-4cde-44b7-9afb-ff1b5886929c service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.152421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.152992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.153266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.153535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] PATH_INFO: `/v3/projects/a3c154cdcab34bd4bc2b5c0e198ccaa1` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.154699 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c154cdcab34bd4bc2b5c0e198ccaa1)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.156586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:58.205637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c154cdcab34bd4bc2b5c0e198ccaa1', 'request_id': 'req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:58.212478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c154cdcab34bd4bc2b5c0e198ccaa1', 'request_id': 'req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:58.218518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c154cdcab34bd4bc2b5c0e198ccaa1', 'request_id': 'req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:58.229772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] The token cache is being invalidate because project a3c154cdcab34bd4bc2b5c0e198ccaa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:58.230064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:58.230804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dc5e3fe3-13f9-43ae-b442-3a41e405cfc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:58.232690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1061/2117] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:58 2022] DELETE /identity/v3/projects/a3c154cdcab34bd4bc2b5c0e198ccaa1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:58.237943 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146f21-f057-4d89-8f98-5152d29c45fa None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.238267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146f21-f057-4d89-8f98-5152d29c45fa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.238543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146f21-f057-4d89-8f98-5152d29c45fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.259123 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b7146f21-f057-4d89-8f98-5152d29c45fa None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:58.260346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b7146f21-f057-4d89-8f98-5152d29c45fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:58.324031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1057/2118] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:58.333190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75314f5d-e0f3-47f4-b3c2-c804c86f5bc5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.333471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75314f5d-e0f3-47f4-b3c2-c804c86f5bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.333754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75314f5d-e0f3-47f4-b3c2-c804c86f5bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.355537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-75314f5d-e0f3-47f4-b3c2-c804c86f5bc5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:58.356678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-75314f5d-e0f3-47f4-b3c2-c804c86f5bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:58.366325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1062/2119] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:58.370740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7146f21-f057-4d89-8f98-5152d29c45fa None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.381135 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7146f21-f057-4d89-8f98-5152d29c45fa None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.386794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.387479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.388252 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.388534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.389194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:58.389500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.391140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:58.404944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1058/2120] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:58.409768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75314f5d-e0f3-47f4-b3c2-c804c86f5bc5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.419672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75314f5d-e0f3-47f4-b3c2-c804c86f5bc5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.425620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b69fc6-d66e-4b49-9106-dc8fcc0e1cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.426252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b69fc6-d66e-4b49-9106-dc8fcc0e1cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.426525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b69fc6-d66e-4b49-9106-dc8fcc0e1cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.426788 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b69fc6-d66e-4b49-9106-dc8fcc0e1cf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.427417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b69fc6-d66e-4b49-9106-dc8fcc0e1cf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.429394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b69fc6-d66e-4b49-9106-dc8fcc0e1cf2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:58.465080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1063/2121] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:58 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:58.468421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.479845 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201998c-03e4-4406-bcf4-68f86f7171c9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.538343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94a59-8d4d-413b-941b-32cdfb7cc41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.538956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94a59-8d4d-413b-941b-32cdfb7cc41e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.539227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94a59-8d4d-413b-941b-32cdfb7cc41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.539490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94a59-8d4d-413b-941b-32cdfb7cc41e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.540117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94a59-8d4d-413b-941b-32cdfb7cc41e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.542022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94a59-8d4d-413b-941b-32cdfb7cc41e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:58.564078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1059/2122] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:58 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:58.568518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b69fc6-d66e-4b49-9106-dc8fcc0e1cf2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.578823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b69fc6-d66e-4b49-9106-dc8fcc0e1cf2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.583997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebb356-8b08-47cc-9e62-f92651f2acf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.584578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebb356-8b08-47cc-9e62-f92651f2acf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.584898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebb356-8b08-47cc-9e62-f92651f2acf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.585173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebb356-8b08-47cc-9e62-f92651f2acf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.585852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebb356-8b08-47cc-9e62-f92651f2acf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.587502 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebb356-8b08-47cc-9e62-f92651f2acf7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:58.598561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1064/2123] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:58.602692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94a59-8d4d-413b-941b-32cdfb7cc41e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.613295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94a59-8d4d-413b-941b-32cdfb7cc41e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.619039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86865e-db55-4238-9069-00caf9842661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.619665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86865e-db55-4238-9069-00caf9842661 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.619947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86865e-db55-4238-9069-00caf9842661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.620204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86865e-db55-4238-9069-00caf9842661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.620843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86865e-db55-4238-9069-00caf9842661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.622497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86865e-db55-4238-9069-00caf9842661 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:58.633576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1060/2124] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:58.637908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebb356-8b08-47cc-9e62-f92651f2acf7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.647866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebb356-8b08-47cc-9e62-f92651f2acf7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.652496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82a3b-c98c-4363-802a-72d7fb9d47e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.653143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82a3b-c98c-4363-802a-72d7fb9d47e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.653420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82a3b-c98c-4363-802a-72d7fb9d47e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.653690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82a3b-c98c-4363-802a-72d7fb9d47e6 admin admin] PATH_INFO: `/v3/projects/e59ac1a8e2f74205a4c5833e66208d30/users/d88ddaa1aaa248a2a5c8773c3f3ae3e0/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.668771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef82a3b-c98c-4363-802a-72d7fb9d47e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e59ac1a8e2f74205a4c5833e66208d30, user_id=d88ddaa1aaa248a2a5c8773c3f3ae3e0, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.671450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef82a3b-c98c-4363-802a-72d7fb9d47e6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:58.686940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1065/2125] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:58 2022] PUT /identity/v3/projects/e59ac1a8e2f74205a4c5833e66208d30/users/d88ddaa1aaa248a2a5c8773c3f3ae3e0/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:58.697824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3395e-f54a-4900-89ee-60fc890c1a98 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.698184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3395e-f54a-4900-89ee-60fc890c1a98 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.698458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3395e-f54a-4900-89ee-60fc890c1a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.727898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ccd3395e-f54a-4900-89ee-60fc890c1a98 None None] MFA Rules not processed for user `d88ddaa1aaa248a2a5c8773c3f3ae3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:58.729139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ccd3395e-f54a-4900-89ee-60fc890c1a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:58.743616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:58.801096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1061/2126] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:36:58.805470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82a3b-c98c-4363-802a-72d7fb9d47e6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.819402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:58.874051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82a3b-c98c-4363-802a-72d7fb9d47e6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.880065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983c9b-97d4-45ee-943f-e9f71837e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.880742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983c9b-97d4-45ee-943f-e9f71837e90b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.881018 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983c9b-97d4-45ee-943f-e9f71837e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.881300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983c9b-97d4-45ee-943f-e9f71837e90b admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.881964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3983c9b-97d4-45ee-943f-e9f71837e90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.883867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3983c9b-97d4-45ee-943f-e9f71837e90b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:58.912806 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1066/2127] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:58 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:58.917429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3395e-f54a-4900-89ee-60fc890c1a98 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:58.928168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3395e-f54a-4900-89ee-60fc890c1a98 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:58.983318 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868cf9d-8f21-491b-9ef0-b029bbd078d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:58.983914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868cf9d-8f21-491b-9ef0-b029bbd078d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:58.984170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868cf9d-8f21-491b-9ef0-b029bbd078d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:58.984438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868cf9d-8f21-491b-9ef0-b029bbd078d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:58.985053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868cf9d-8f21-491b-9ef0-b029bbd078d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:58.987188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868cf9d-8f21-491b-9ef0-b029bbd078d4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:59.010406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1062/2128] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:58 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:59.016218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983c9b-97d4-45ee-943f-e9f71837e90b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:59.027273 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983c9b-97d4-45ee-943f-e9f71837e90b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:59.032626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5154d6-972e-48d0-92b4-ac24dd443115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:59.033229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5154d6-972e-48d0-92b4-ac24dd443115 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:59.033511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5154d6-972e-48d0-92b4-ac24dd443115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:59.033785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5154d6-972e-48d0-92b4-ac24dd443115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:59.035170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5154d6-972e-48d0-92b4-ac24dd443115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:59.036893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5154d6-972e-48d0-92b4-ac24dd443115 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:59.047880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1067/2129] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:59.052335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868cf9d-8f21-491b-9ef0-b029bbd078d4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:59.062925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868cf9d-8f21-491b-9ef0-b029bbd078d4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:59.068159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a42df-8f7c-4aad-bd46-8c385e4c5287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:59.068807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a42df-8f7c-4aad-bd46-8c385e4c5287 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:59.069081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a42df-8f7c-4aad-bd46-8c385e4c5287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:59.069347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a42df-8f7c-4aad-bd46-8c385e4c5287 admin admin] PATH_INFO: `/v3/projects/f84cb47b78954755b78e0437ef43a516/users/893d28bc89f44bafac58b9247c8e3130/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:59.084542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a42df-8f7c-4aad-bd46-8c385e4c5287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84cb47b78954755b78e0437ef43a516, user_id=893d28bc89f44bafac58b9247c8e3130, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:59.087379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a42df-8f7c-4aad-bd46-8c385e4c5287 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:59.104094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1063/2130] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:59 2022] PUT /identity/v3/projects/f84cb47b78954755b78e0437ef43a516/users/893d28bc89f44bafac58b9247c8e3130/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:59.108414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5154d6-972e-48d0-92b4-ac24dd443115 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:59.118929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5154d6-972e-48d0-92b4-ac24dd443115 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:59.182324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28675bb-22c6-4d77-b291-1fddabddb3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:59.183082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28675bb-22c6-4d77-b291-1fddabddb3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:59.183408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28675bb-22c6-4d77-b291-1fddabddb3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:59.183754 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28675bb-22c6-4d77-b291-1fddabddb3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:59.184725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28675bb-22c6-4d77-b291-1fddabddb3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:59.187117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28675bb-22c6-4d77-b291-1fddabddb3a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:59.202820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1068/2131] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:59.208616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a42df-8f7c-4aad-bd46-8c385e4c5287 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:59.219357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a42df-8f7c-4aad-bd46-8c385e4c5287 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:59.224585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:59.225211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:59.225520 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:59.225797 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] PATH_INFO: `/v3/projects/f84cb47b78954755b78e0437ef43a516/users/893d28bc89f44bafac58b9247c8e3130/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:59.227815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84cb47b78954755b78e0437ef43a516, user_id=893d28bc89f44bafac58b9247c8e3130, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:59.230501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:59.241324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1064/2132] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:59 2022] PUT /identity/v3/projects/f84cb47b78954755b78e0437ef43a516/users/893d28bc89f44bafac58b9247c8e3130/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:59.251914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de7e28-c157-49d1-b085-d289b69d7763 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:59.252198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de7e28-c157-49d1-b085-d289b69d7763 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:59.252476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de7e28-c157-49d1-b085-d289b69d7763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:59.281951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f0de7e28-c157-49d1-b085-d289b69d7763 None None] MFA Rules not processed for user `893d28bc89f44bafac58b9247c8e3130`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:59.283437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f0de7e28-c157-49d1-b085-d289b69d7763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:59.357836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1069/2133] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:36:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:59.369396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:59.371513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:59.431264 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9724-5aba-4c75-b575-abb8034a08b6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:59.435496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:59.436132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:59.436410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:59.436678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:59.447904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:59.449347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:59.459831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1065/2134] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:59.548679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de7e28-c157-49d1-b085-d289b69d7763 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:59.559671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de7e28-c157-49d1-b085-d289b69d7763 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:59.564060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087d5e3-b380-40e0-be30-928ae21314da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:59.564654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087d5e3-b380-40e0-be30-928ae21314da service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:59.564924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087d5e3-b380-40e0-be30-928ae21314da service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:59.565202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087d5e3-b380-40e0-be30-928ae21314da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:59.570705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:59.623322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087d5e3-b380-40e0-be30-928ae21314da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:59.624760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087d5e3-b380-40e0-be30-928ae21314da service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:59.633691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1070/2135] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:59.936175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:59.937913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:59.953046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:00.006249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b530070-3e9f-46cd-a771-fddba8095004 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:00.010725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:00.011372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:00.011599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:00.011892 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] PATH_INFO: `/v3/users/1f7765a148a64712bcf65ad912b31aef` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:00.012991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7765a148a64712bcf65ad912b31aef)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:00.014991 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:00.067082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7765a148a64712bcf65ad912b31aef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.067861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7765a148a64712bcf65ad912b31aef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.081808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7765a148a64712bcf65ad912b31aef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.089486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7765a148a64712bcf65ad912b31aef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.090225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7765a148a64712bcf65ad912b31aef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.105398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1066/2136] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:59 2022] DELETE /identity/v3/users/1f7765a148a64712bcf65ad912b31aef => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:00.110071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462467c-deb9-4bed-8557-70d96493fc81 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:00.110588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462467c-deb9-4bed-8557-70d96493fc81 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:00.111092 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462467c-deb9-4bed-8557-70d96493fc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:00.131016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0462467c-deb9-4bed-8557-70d96493fc81 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:00.132210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0462467c-deb9-4bed-8557-70d96493fc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:00.193085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1071/2137] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:00.203530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:00.204939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:00.263597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a562af6-090b-4a2b-a976-f96944e89782 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:00.268403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbb4a5-d775-47e4-9a54-05ad4910ea90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:00.269047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb4a5-d775-47e4-9a54-05ad4910ea90 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:00.269325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb4a5-d775-47e4-9a54-05ad4910ea90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:00.269592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb4a5-d775-47e4-9a54-05ad4910ea90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:00.279918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbb4a5-d775-47e4-9a54-05ad4910ea90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:00.281360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbb4a5-d775-47e4-9a54-05ad4910ea90 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:00.291700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1067/2138] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:00.579258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462467c-deb9-4bed-8557-70d96493fc81 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:00.581316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0462467c-deb9-4bed-8557-70d96493fc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:00.593416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462467c-deb9-4bed-8557-70d96493fc81 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:00.598749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:00.599396 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:00.599696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:00.599951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] PATH_INFO: `/v3/projects/81efee57dccc4087aa35f43660dee692` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:00.601108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=81efee57dccc4087aa35f43660dee692)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:00.603059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:00.654311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81efee57dccc4087aa35f43660dee692', 'request_id': 'req-f5930501-09fb-4504-b7dc-9c5a7305f54f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.662916 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81efee57dccc4087aa35f43660dee692', 'request_id': 'req-f5930501-09fb-4504-b7dc-9c5a7305f54f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.669732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81efee57dccc4087aa35f43660dee692', 'request_id': 'req-f5930501-09fb-4504-b7dc-9c5a7305f54f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.681384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] The token cache is being invalidate because project 81efee57dccc4087aa35f43660dee692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:00.681700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.682387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.683918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1072/2139] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:00 2022] DELETE /identity/v3/projects/81efee57dccc4087aa35f43660dee692 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:00.688188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbb4a5-d775-47e4-9a54-05ad4910ea90 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:00.698971 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbb4a5-d775-47e4-9a54-05ad4910ea90 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:00.730638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:00.763116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:00.763727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:00.763996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:00.764280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] PATH_INFO: `/v3/users/6d73a25467af4da4ae94fb26c0c278bb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:00.765412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d73a25467af4da4ae94fb26c0c278bb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:00.767639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:00.820364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d73a25467af4da4ae94fb26c0c278bb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.821042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d73a25467af4da4ae94fb26c0c278bb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.837208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d73a25467af4da4ae94fb26c0c278bb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.844838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d73a25467af4da4ae94fb26c0c278bb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.846779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5ab4677c-f596-488b-aae1-bb872fd54f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d73a25467af4da4ae94fb26c0c278bb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:00.866076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1068/2140] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:00 2022] DELETE /identity/v3/users/6d73a25467af4da4ae94fb26c0c278bb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:01.190526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:01.191977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:01.217655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5930501-09fb-4504-b7dc-9c5a7305f54f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:01.257768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:01.291584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:01.292402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.292750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.293017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] PATH_INFO: `/v3/projects/701ac49cb9164da7abc11e823e175256` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.294610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=701ac49cb9164da7abc11e823e175256)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:01.297841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:01.357040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701ac49cb9164da7abc11e823e175256', 'request_id': 'req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:01.366617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701ac49cb9164da7abc11e823e175256', 'request_id': 'req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:01.373456 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701ac49cb9164da7abc11e823e175256', 'request_id': 'req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:01.384276 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] The token cache is being invalidate because project 701ac49cb9164da7abc11e823e175256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:01.384560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:01.385197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a023fcd3-1729-434e-8b8e-ad3f7cc4fe09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:01.386797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1073/2141] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:01 2022] DELETE /identity/v3/projects/701ac49cb9164da7abc11e823e175256 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:01.392763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123950f7-8b1c-49ef-9002-a627a15c1aeb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.393605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123950f7-8b1c-49ef-9002-a627a15c1aeb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.393902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123950f7-8b1c-49ef-9002-a627a15c1aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.413387 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-123950f7-8b1c-49ef-9002-a627a15c1aeb None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:01.414613 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-123950f7-8b1c-49ef-9002-a627a15c1aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:01.474275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1069/2142] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:01.483382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff234-34d2-482b-98b2-7c6749ce321d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.483663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff234-34d2-482b-98b2-7c6749ce321d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.483937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff234-34d2-482b-98b2-7c6749ce321d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.504952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-74cff234-34d2-482b-98b2-7c6749ce321d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:01.506186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-74cff234-34d2-482b-98b2-7c6749ce321d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:01.514065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1074/2143] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:01.518410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123950f7-8b1c-49ef-9002-a627a15c1aeb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:01.528282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123950f7-8b1c-49ef-9002-a627a15c1aeb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:01.533377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:01.533978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.534295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.534563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.535213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:01.535519 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:01.537108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:01.553881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1070/2144] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:01.558383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cff234-34d2-482b-98b2-7c6749ce321d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:01.569358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cff234-34d2-482b-98b2-7c6749ce321d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:01.575415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917497a0-db81-49b3-9d06-929f9081042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:01.576017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917497a0-db81-49b3-9d06-929f9081042b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.576285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917497a0-db81-49b3-9d06-929f9081042b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.576551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917497a0-db81-49b3-9d06-929f9081042b admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.577216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917497a0-db81-49b3-9d06-929f9081042b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:01.579179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917497a0-db81-49b3-9d06-929f9081042b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:01.612931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1075/2145] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:01 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:01.619582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:01.632551 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f0f0c-69b8-4345-9fdf-868edfb6cff9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:01.671793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:01.694146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788082e-dccc-4857-8d13-cfd483ada5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:01.694846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788082e-dccc-4857-8d13-cfd483ada5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.695165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788082e-dccc-4857-8d13-cfd483ada5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.695457 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788082e-dccc-4857-8d13-cfd483ada5c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.696158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788082e-dccc-4857-8d13-cfd483ada5c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:01.698357 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788082e-dccc-4857-8d13-cfd483ada5c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:01.721982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1071/2146] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:01 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:01.726283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917497a0-db81-49b3-9d06-929f9081042b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:01.737457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917497a0-db81-49b3-9d06-929f9081042b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:01.742951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b96969-b6f9-43cf-8b55-b3e8d7702e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:01.743559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b96969-b6f9-43cf-8b55-b3e8d7702e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.743882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b96969-b6f9-43cf-8b55-b3e8d7702e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.744170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b96969-b6f9-43cf-8b55-b3e8d7702e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.744799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b96969-b6f9-43cf-8b55-b3e8d7702e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:01.746470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b96969-b6f9-43cf-8b55-b3e8d7702e79 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:01.757613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1076/2147] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:01.761777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788082e-dccc-4857-8d13-cfd483ada5c6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:01.772430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788082e-dccc-4857-8d13-cfd483ada5c6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:01.777531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9776e-8aa6-46e7-abfb-999394a28844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:01.778137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9776e-8aa6-46e7-abfb-999394a28844 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.778417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9776e-8aa6-46e7-abfb-999394a28844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.778681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9776e-8aa6-46e7-abfb-999394a28844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.779311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9776e-8aa6-46e7-abfb-999394a28844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:01.780926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9776e-8aa6-46e7-abfb-999394a28844 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:01.791361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1072/2148] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:01.795417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b96969-b6f9-43cf-8b55-b3e8d7702e79 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:01.805881 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b96969-b6f9-43cf-8b55-b3e8d7702e79 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:01.811247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:01.811893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.812169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.812431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] PATH_INFO: `/v3/projects/ab8f66bca550493798a7aee3a189fb8b/users/e4627110f092433694313fea1847475f/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.828111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab8f66bca550493798a7aee3a189fb8b, user_id=e4627110f092433694313fea1847475f, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:01.830945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:01.845733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1077/2149] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:01 2022] PUT /identity/v3/projects/ab8f66bca550493798a7aee3a189fb8b/users/e4627110f092433694313fea1847475f/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:01.857699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ad38f-9a9a-4d02-b972-f187570d6405 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:01.858144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ad38f-9a9a-4d02-b972-f187570d6405 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:01.858511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ad38f-9a9a-4d02-b972-f187570d6405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:01.889626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f89ad38f-9a9a-4d02-b972-f187570d6405 None None] MFA Rules not processed for user `e4627110f092433694313fea1847475f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:01.890896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f89ad38f-9a9a-4d02-b972-f187570d6405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:01.959006 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1073/2150] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:01.970948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:01.973264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:02.032771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947aa410-68f8-4d51-b22b-46c24535dd86 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:02.036553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b94ea4-1a21-487f-a22d-1a00be3761eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:02.037168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94ea4-1a21-487f-a22d-1a00be3761eb service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:02.037448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94ea4-1a21-487f-a22d-1a00be3761eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:02.037723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94ea4-1a21-487f-a22d-1a00be3761eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:02.046260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b94ea4-1a21-487f-a22d-1a00be3761eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:02.047730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b94ea4-1a21-487f-a22d-1a00be3761eb service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:02.052735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:02.057639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1078/2151] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:11.184494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ad38f-9a9a-4d02-b972-f187570d6405 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:11.185924 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f89ad38f-9a9a-4d02-b972-f187570d6405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:11.254949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ad38f-9a9a-4d02-b972-f187570d6405 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:11.259820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:11.260425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:11.260698 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:11.260964 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] PATH_INFO: `/v3/users/d88ddaa1aaa248a2a5c8773c3f3ae3e0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:11.262130 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88ddaa1aaa248a2a5c8773c3f3ae3e0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:11.264036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:11.321265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88ddaa1aaa248a2a5c8773c3f3ae3e0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.321922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88ddaa1aaa248a2a5c8773c3f3ae3e0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.323786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:11.335673 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88ddaa1aaa248a2a5c8773c3f3ae3e0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.342408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88ddaa1aaa248a2a5c8773c3f3ae3e0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.343234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88ddaa1aaa248a2a5c8773c3f3ae3e0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.357705 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1074/2152] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:11 2022] DELETE /identity/v3/users/d88ddaa1aaa248a2a5c8773c3f3ae3e0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:11.362265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948db44-32bd-4dd8-87f1-a7a635ea0161 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:11.362786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948db44-32bd-4dd8-87f1-a7a635ea0161 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:11.362786 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948db44-32bd-4dd8-87f1-a7a635ea0161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:11.384070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4948db44-32bd-4dd8-87f1-a7a635ea0161 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:11.385265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4948db44-32bd-4dd8-87f1-a7a635ea0161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:11.452725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1079/2153] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:11.463243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:11.464828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:11.523125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc94e5-56a1-4ddd-9762-ca6e8b993875 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:11.526972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1011e7-589a-44b0-84f7-284451dba40a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:11.527571 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1011e7-589a-44b0-84f7-284451dba40a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:11.527842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1011e7-589a-44b0-84f7-284451dba40a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:11.528104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1011e7-589a-44b0-84f7-284451dba40a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:11.537240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1011e7-589a-44b0-84f7-284451dba40a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:11.538849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1011e7-589a-44b0-84f7-284451dba40a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:11.548240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1075/2154] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:11.838873 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948db44-32bd-4dd8-87f1-a7a635ea0161 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:11.840520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4948db44-32bd-4dd8-87f1-a7a635ea0161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:11.855776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948db44-32bd-4dd8-87f1-a7a635ea0161 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:11.860768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:11.861886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:11.861886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:11.861886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] PATH_INFO: `/v3/projects/e59ac1a8e2f74205a4c5833e66208d30` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:11.863119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e59ac1a8e2f74205a4c5833e66208d30)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:11.865006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:11.916373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e59ac1a8e2f74205a4c5833e66208d30', 'request_id': 'req-58ad847f-9db5-4037-9bbf-15261aa76ec0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.926691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e59ac1a8e2f74205a4c5833e66208d30', 'request_id': 'req-58ad847f-9db5-4037-9bbf-15261aa76ec0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.934562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e59ac1a8e2f74205a4c5833e66208d30', 'request_id': 'req-58ad847f-9db5-4037-9bbf-15261aa76ec0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.945839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] The token cache is being invalidate because project e59ac1a8e2f74205a4c5833e66208d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:11.946279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.947102 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:11.948688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1080/2155] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:11 2022] DELETE /identity/v3/projects/e59ac1a8e2f74205a4c5833e66208d30 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:11.952772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1011e7-589a-44b0-84f7-284451dba40a service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:11.963526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1011e7-589a-44b0-84f7-284451dba40a service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:12.022079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:12.023355 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.023637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.023911 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] PATH_INFO: `/v3/users/893d28bc89f44bafac58b9247c8e3130` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.025039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] RBAC: Authorizing `identity:delete_user(user_id=893d28bc89f44bafac58b9247c8e3130)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:12.027051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:12.076718 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893d28bc89f44bafac58b9247c8e3130'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.077372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893d28bc89f44bafac58b9247c8e3130'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.091638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893d28bc89f44bafac58b9247c8e3130'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.097009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:12.099663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893d28bc89f44bafac58b9247c8e3130'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.101011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33a31411-4560-409b-b6ef-2e1b3b9bb18c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893d28bc89f44bafac58b9247c8e3130'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.115070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1076/2156] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:11 2022] DELETE /identity/v3/users/893d28bc89f44bafac58b9247c8e3130 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:12.409643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:12.411124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:12.425984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad847f-9db5-4037-9bbf-15261aa76ec0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:12.485603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:12.486259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.486531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.486798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] PATH_INFO: `/v3/projects/f84cb47b78954755b78e0437ef43a516` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.487898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84cb47b78954755b78e0437ef43a516)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:12.489823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:12.523130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:12.541147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84cb47b78954755b78e0437ef43a516', 'request_id': 'req-82b319cf-313d-4ed3-bb92-84c7905102a4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.550543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84cb47b78954755b78e0437ef43a516', 'request_id': 'req-82b319cf-313d-4ed3-bb92-84c7905102a4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.557005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84cb47b78954755b78e0437ef43a516', 'request_id': 'req-82b319cf-313d-4ed3-bb92-84c7905102a4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.567733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] The token cache is being invalidate because project f84cb47b78954755b78e0437ef43a516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:12.568017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.568759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-82b319cf-313d-4ed3-bb92-84c7905102a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:12.570581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1081/2157] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:12 2022] DELETE /identity/v3/projects/f84cb47b78954755b78e0437ef43a516 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:12.575691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be855-4ad1-4b96-baa5-26f3cafa5cb4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.576172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be855-4ad1-4b96-baa5-26f3cafa5cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.576295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be855-4ad1-4b96-baa5-26f3cafa5cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.597784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3b4be855-4ad1-4b96-baa5-26f3cafa5cb4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:12.599222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b4be855-4ad1-4b96-baa5-26f3cafa5cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:12.667063 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1077/2158] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:12.675235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa320a-564d-4fdd-996a-1509abc5b83e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.675510 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa320a-564d-4fdd-996a-1509abc5b83e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.675781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa320a-564d-4fdd-996a-1509abc5b83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.696159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-08fa320a-564d-4fdd-996a-1509abc5b83e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:12.697426 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-08fa320a-564d-4fdd-996a-1509abc5b83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:12.706350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1082/2159] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:12.711014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4be855-4ad1-4b96-baa5-26f3cafa5cb4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:12.721043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4be855-4ad1-4b96-baa5-26f3cafa5cb4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:12.727330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:12.727951 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.728225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.728492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.729134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:12.729446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:12.731079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:12.743806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1078/2160] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:12.749882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa320a-564d-4fdd-996a-1509abc5b83e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:12.759634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa320a-564d-4fdd-996a-1509abc5b83e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:12.766144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cd575-b7ff-4c8e-a523-97796d8d0020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:12.766742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd575-b7ff-4c8e-a523-97796d8d0020 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.767011 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd575-b7ff-4c8e-a523-97796d8d0020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.767283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd575-b7ff-4c8e-a523-97796d8d0020 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.767898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cd575-b7ff-4c8e-a523-97796d8d0020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:12.769790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cd575-b7ff-4c8e-a523-97796d8d0020 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:12.799418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1083/2161] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:12 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:12.803970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:12.814610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83ce9-b1d0-425c-921c-eb26d33742c1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:12.873136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208a75f-5ee9-4d92-a19a-32ee38caeb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:12.873731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208a75f-5ee9-4d92-a19a-32ee38caeb19 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.874012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208a75f-5ee9-4d92-a19a-32ee38caeb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.874316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208a75f-5ee9-4d92-a19a-32ee38caeb19 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.874916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208a75f-5ee9-4d92-a19a-32ee38caeb19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:12.876847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208a75f-5ee9-4d92-a19a-32ee38caeb19 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:12.899161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1079/2162] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:12 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:12.903980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cd575-b7ff-4c8e-a523-97796d8d0020 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:12.917186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cd575-b7ff-4c8e-a523-97796d8d0020 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:12.923059 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bcd8a-fcb8-4247-8752-bef3e0360b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:12.923667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bcd8a-fcb8-4247-8752-bef3e0360b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.923946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bcd8a-fcb8-4247-8752-bef3e0360b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.924243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bcd8a-fcb8-4247-8752-bef3e0360b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.924886 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bcd8a-fcb8-4247-8752-bef3e0360b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:12.926607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bcd8a-fcb8-4247-8752-bef3e0360b6e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:12.936925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1084/2163] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:12.941193 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208a75f-5ee9-4d92-a19a-32ee38caeb19 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:12.952103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208a75f-5ee9-4d92-a19a-32ee38caeb19 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:12.957503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f22e75-48c2-489d-9482-2b130baaf9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:12.958132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22e75-48c2-489d-9482-2b130baaf9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.958393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22e75-48c2-489d-9482-2b130baaf9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.958646 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22e75-48c2-489d-9482-2b130baaf9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.959307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f22e75-48c2-489d-9482-2b130baaf9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:12.961057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f22e75-48c2-489d-9482-2b130baaf9e5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:12.972303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1080/2164] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:12.976871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bcd8a-fcb8-4247-8752-bef3e0360b6e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:12.987133 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bcd8a-fcb8-4247-8752-bef3e0360b6e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:12.992286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023b70-a886-4a34-a8ed-e6de6e1baf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:12.992930 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023b70-a886-4a34-a8ed-e6de6e1baf5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.993208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023b70-a886-4a34-a8ed-e6de6e1baf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.993484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023b70-a886-4a34-a8ed-e6de6e1baf5d admin admin] PATH_INFO: `/v3/projects/bc354d81968945bf9375ab7ffd7bde0f/users/b864171c26b5484087478a348c6059c2/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.008443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e023b70-a886-4a34-a8ed-e6de6e1baf5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc354d81968945bf9375ab7ffd7bde0f, user_id=b864171c26b5484087478a348c6059c2, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.011129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e023b70-a886-4a34-a8ed-e6de6e1baf5d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.026432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1085/2165] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:12 2022] PUT /identity/v3/projects/bc354d81968945bf9375ab7ffd7bde0f/users/b864171c26b5484087478a348c6059c2/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:13.036886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc743760-39e6-4b6e-a96f-b33fc710e903 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.037150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc743760-39e6-4b6e-a96f-b33fc710e903 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.037447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc743760-39e6-4b6e-a96f-b33fc710e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.065866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dc743760-39e6-4b6e-a96f-b33fc710e903 None None] MFA Rules not processed for user `b864171c26b5484087478a348c6059c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:13.067115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc743760-39e6-4b6e-a96f-b33fc710e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:13.115155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:13.143178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1081/2166] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:13.147908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023b70-a886-4a34-a8ed-e6de6e1baf5d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.213350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023b70-a886-4a34-a8ed-e6de6e1baf5d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.218588 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1d6e2-636a-4700-a340-24fcd5ec5c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.219190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1d6e2-636a-4700-a340-24fcd5ec5c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.219499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1d6e2-636a-4700-a340-24fcd5ec5c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.219773 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1d6e2-636a-4700-a340-24fcd5ec5c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.220405 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1d6e2-636a-4700-a340-24fcd5ec5c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.222664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1d6e2-636a-4700-a340-24fcd5ec5c89 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.252252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1086/2167] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:13 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:13.256542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc743760-39e6-4b6e-a96f-b33fc710e903 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.267376 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc743760-39e6-4b6e-a96f-b33fc710e903 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.328426 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034b89-8b78-47e3-a426-849711aee55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.329024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034b89-8b78-47e3-a426-849711aee55a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.329298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034b89-8b78-47e3-a426-849711aee55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.329589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034b89-8b78-47e3-a426-849711aee55a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.330253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db034b89-8b78-47e3-a426-849711aee55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.332149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db034b89-8b78-47e3-a426-849711aee55a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.356804 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1082/2168] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:13 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:37:13.360920 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1d6e2-636a-4700-a340-24fcd5ec5c89 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.371980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1d6e2-636a-4700-a340-24fcd5ec5c89 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.377388 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3915a-aa44-452a-98a7-7695c2a56b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.377989 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3915a-aa44-452a-98a7-7695c2a56b32 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.378307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3915a-aa44-452a-98a7-7695c2a56b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.378553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3915a-aa44-452a-98a7-7695c2a56b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.379214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3915a-aa44-452a-98a7-7695c2a56b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.380827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3915a-aa44-452a-98a7-7695c2a56b32 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.392708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1087/2169] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:13.395579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034b89-8b78-47e3-a426-849711aee55a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.406544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034b89-8b78-47e3-a426-849711aee55a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.412410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a7dc0-7b8e-436b-9bf5-fbdb9418e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.413036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7dc0-7b8e-436b-9bf5-fbdb9418e64f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.413321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7dc0-7b8e-436b-9bf5-fbdb9418e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.413598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7dc0-7b8e-436b-9bf5-fbdb9418e64f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.414273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a7dc0-7b8e-436b-9bf5-fbdb9418e64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.416056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a7dc0-7b8e-436b-9bf5-fbdb9418e64f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.426777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1083/2170] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:13.430946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3915a-aa44-452a-98a7-7695c2a56b32 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.441511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3915a-aa44-452a-98a7-7695c2a56b32 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.446745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47422244-9da6-4b8d-8e5f-f85cbb954851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.447384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47422244-9da6-4b8d-8e5f-f85cbb954851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.447667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47422244-9da6-4b8d-8e5f-f85cbb954851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.447935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47422244-9da6-4b8d-8e5f-f85cbb954851 admin admin] PATH_INFO: `/v3/projects/913664a154df4f33bc5ada5b54298a50/users/48f404324e17499da2b29cc8804b6eb7/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.467023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47422244-9da6-4b8d-8e5f-f85cbb954851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913664a154df4f33bc5ada5b54298a50, user_id=48f404324e17499da2b29cc8804b6eb7, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.469607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47422244-9da6-4b8d-8e5f-f85cbb954851 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.481846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:13.487309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1088/2171] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:13 2022] PUT /identity/v3/projects/913664a154df4f33bc5ada5b54298a50/users/48f404324e17499da2b29cc8804b6eb7/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:13.499618 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c28df-1e6e-4210-b210-925d50f84955 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.499925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c28df-1e6e-4210-b210-925d50f84955 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.500167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c28df-1e6e-4210-b210-925d50f84955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.529483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-583c28df-1e6e-4210-b210-925d50f84955 None None] MFA Rules not processed for user `48f404324e17499da2b29cc8804b6eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:13.530725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-583c28df-1e6e-4210-b210-925d50f84955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:13.599760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1084/2172] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:13.604568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47422244-9da6-4b8d-8e5f-f85cbb954851 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.670057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47422244-9da6-4b8d-8e5f-f85cbb954851 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.676281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38682d5d-8da8-41b1-a678-96c87eb320a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.676899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38682d5d-8da8-41b1-a678-96c87eb320a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.677175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38682d5d-8da8-41b1-a678-96c87eb320a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.677443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38682d5d-8da8-41b1-a678-96c87eb320a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.678080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38682d5d-8da8-41b1-a678-96c87eb320a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.680129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38682d5d-8da8-41b1-a678-96c87eb320a7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.709373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1089/2173] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:13 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:13.715253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c28df-1e6e-4210-b210-925d50f84955 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.732946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c28df-1e6e-4210-b210-925d50f84955 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.786204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:13.797150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728b079-b1b2-400e-b761-5a74de94966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.797758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b079-b1b2-400e-b761-5a74de94966a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.798040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b079-b1b2-400e-b761-5a74de94966a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.798347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b079-b1b2-400e-b761-5a74de94966a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.798975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728b079-b1b2-400e-b761-5a74de94966a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.800937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728b079-b1b2-400e-b761-5a74de94966a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.824369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1085/2174] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:13 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:13.828535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38682d5d-8da8-41b1-a678-96c87eb320a7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.839610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38682d5d-8da8-41b1-a678-96c87eb320a7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.845055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4ed8b-edd1-4f99-b271-9c3d8f23a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.845648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4ed8b-edd1-4f99-b271-9c3d8f23a48d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.845945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4ed8b-edd1-4f99-b271-9c3d8f23a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.846522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4ed8b-edd1-4f99-b271-9c3d8f23a48d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.847201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4ed8b-edd1-4f99-b271-9c3d8f23a48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.848785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4ed8b-edd1-4f99-b271-9c3d8f23a48d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.860847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1090/2175] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:13.865058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728b079-b1b2-400e-b761-5a74de94966a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.875190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728b079-b1b2-400e-b761-5a74de94966a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.879968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b332398-4076-406c-aa40-24a11ee0dbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.880606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b332398-4076-406c-aa40-24a11ee0dbb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.880896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b332398-4076-406c-aa40-24a11ee0dbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.881166 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b332398-4076-406c-aa40-24a11ee0dbb3 admin admin] PATH_INFO: `/v3/projects/331ed54c9e064ae4a9c09f8a78e3d799/users/e2ba197d43ee48289b4b53c353ba935a/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.898801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b332398-4076-406c-aa40-24a11ee0dbb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=331ed54c9e064ae4a9c09f8a78e3d799, user_id=e2ba197d43ee48289b4b53c353ba935a, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.901463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b332398-4076-406c-aa40-24a11ee0dbb3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.917050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1086/2176] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:13 2022] PUT /identity/v3/projects/331ed54c9e064ae4a9c09f8a78e3d799/users/e2ba197d43ee48289b4b53c353ba935a/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:13.921304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4ed8b-edd1-4f99-b271-9c3d8f23a48d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.931476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4ed8b-edd1-4f99-b271-9c3d8f23a48d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.992115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aba96d-769c-4110-9d1a-998e0348619e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.992630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba96d-769c-4110-9d1a-998e0348619e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.992909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba96d-769c-4110-9d1a-998e0348619e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.993161 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba96d-769c-4110-9d1a-998e0348619e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.993802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aba96d-769c-4110-9d1a-998e0348619e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.995449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aba96d-769c-4110-9d1a-998e0348619e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:14.005205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1091/2177] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:14.010109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b332398-4076-406c-aa40-24a11ee0dbb3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:14.020585 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b332398-4076-406c-aa40-24a11ee0dbb3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:14.026751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:14.027385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:14.027657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:14.027933 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] PATH_INFO: `/v3/projects/331ed54c9e064ae4a9c09f8a78e3d799/users/e2ba197d43ee48289b4b53c353ba935a/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:14.030057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=331ed54c9e064ae4a9c09f8a78e3d799, user_id=e2ba197d43ee48289b4b53c353ba935a, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:14.032709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:14.044249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1087/2178] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:14 2022] PUT /identity/v3/projects/331ed54c9e064ae4a9c09f8a78e3d799/users/e2ba197d43ee48289b4b53c353ba935a/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:14.054948 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b105ac1-5557-454e-8868-2d6664209795 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:14.055228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b105ac1-5557-454e-8868-2d6664209795 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:14.055496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b105ac1-5557-454e-8868-2d6664209795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:14.086120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6b105ac1-5557-454e-8868-2d6664209795 None None] MFA Rules not processed for user `e2ba197d43ee48289b4b53c353ba935a`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:14.087360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b105ac1-5557-454e-8868-2d6664209795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:14.148429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:14.162533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1092/2179] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:14.170771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:14.173561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:14.236973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c7873-ed2f-4153-a3c4-bafb1cff2edf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:14.241710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20314076-5d7e-48ed-81ec-81ce05c05d86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:14.242405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314076-5d7e-48ed-81ec-81ce05c05d86 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:14.242615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314076-5d7e-48ed-81ec-81ce05c05d86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:14.242877 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314076-5d7e-48ed-81ec-81ce05c05d86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:14.254369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20314076-5d7e-48ed-81ec-81ce05c05d86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:14.255937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20314076-5d7e-48ed-81ec-81ce05c05d86 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:14.265890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1088/2180] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:14.310132 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b105ac1-5557-454e-8868-2d6664209795 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:14.311841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b105ac1-5557-454e-8868-2d6664209795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:14.325238 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b105ac1-5557-454e-8868-2d6664209795 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:14.329659 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f2f1a-a8f4-41e9-99e2-cf2af70b9315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:14.330290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f2f1a-a8f4-41e9-99e2-cf2af70b9315 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:14.330566 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f2f1a-a8f4-41e9-99e2-cf2af70b9315 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:14.330857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f2f1a-a8f4-41e9-99e2-cf2af70b9315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:14.393642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f2f1a-a8f4-41e9-99e2-cf2af70b9315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:14.395134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f2f1a-a8f4-41e9-99e2-cf2af70b9315 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:14.406228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1093/2181] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:19.461196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20314076-5d7e-48ed-81ec-81ce05c05d86 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:19.530190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20314076-5d7e-48ed-81ec-81ce05c05d86 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:19.536136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:19.536786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:19.537057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:19.537329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] PATH_INFO: `/v3/users/b864171c26b5484087478a348c6059c2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:19.538619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b864171c26b5484087478a348c6059c2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:19.540592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:19.589021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b864171c26b5484087478a348c6059c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:19.589662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b864171c26b5484087478a348c6059c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:19.591639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:19.604656 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b864171c26b5484087478a348c6059c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:19.611460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b864171c26b5484087478a348c6059c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:19.612201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b864171c26b5484087478a348c6059c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:19.626323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1089/2182] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:19 2022] DELETE /identity/v3/users/b864171c26b5484087478a348c6059c2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:19.631046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c66a-f657-4157-8cc1-f0d79e7e1357 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:19.631552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c66a-f657-4157-8cc1-f0d79e7e1357 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:19.631832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c66a-f657-4157-8cc1-f0d79e7e1357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:19.651990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-baa8c66a-f657-4157-8cc1-f0d79e7e1357 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:19.653202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-baa8c66a-f657-4157-8cc1-f0d79e7e1357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:19.718002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1094/2183] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:19.730121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:19.732347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:19.796983 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845598b-a5ba-4423-8351-7e5e97351b27 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:19.801459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865d34e-7e8e-48fa-9bc1-40562b280a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:19.802058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865d34e-7e8e-48fa-9bc1-40562b280a4f service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:19.802371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865d34e-7e8e-48fa-9bc1-40562b280a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:19.802644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865d34e-7e8e-48fa-9bc1-40562b280a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:19.812223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865d34e-7e8e-48fa-9bc1-40562b280a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:19.813659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865d34e-7e8e-48fa-9bc1-40562b280a4f service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:19.823121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1090/2184] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:20.117772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8c66a-f657-4157-8cc1-f0d79e7e1357 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:20.119223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-baa8c66a-f657-4157-8cc1-f0d79e7e1357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:20.131376 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8c66a-f657-4157-8cc1-f0d79e7e1357 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:20.136414 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:20.136716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:20.137008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:20.137250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] PATH_INFO: `/v3/projects/bc354d81968945bf9375ab7ffd7bde0f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:20.138492 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc354d81968945bf9375ab7ffd7bde0f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:20.140380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:20.189369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc354d81968945bf9375ab7ffd7bde0f', 'request_id': 'req-aa0d502f-67c2-4686-9413-fe391ca61d3a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.197783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc354d81968945bf9375ab7ffd7bde0f', 'request_id': 'req-aa0d502f-67c2-4686-9413-fe391ca61d3a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.204237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc354d81968945bf9375ab7ffd7bde0f', 'request_id': 'req-aa0d502f-67c2-4686-9413-fe391ca61d3a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.215864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] The token cache is being invalidate because project bc354d81968945bf9375ab7ffd7bde0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:20.216150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.216858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.218473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1095/2185] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:20 2022] DELETE /identity/v3/projects/bc354d81968945bf9375ab7ffd7bde0f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:20.222985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865d34e-7e8e-48fa-9bc1-40562b280a4f service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:20.232881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865d34e-7e8e-48fa-9bc1-40562b280a4f service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:20.290869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:20.291506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:20.291775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:20.292046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] PATH_INFO: `/v3/users/48f404324e17499da2b29cc8804b6eb7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:20.293199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f404324e17499da2b29cc8804b6eb7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:20.295158 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:20.348393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f404324e17499da2b29cc8804b6eb7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.348393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f404324e17499da2b29cc8804b6eb7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.359848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f404324e17499da2b29cc8804b6eb7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.366523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:20.368643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f404324e17499da2b29cc8804b6eb7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.369481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f404324e17499da2b29cc8804b6eb7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.383651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1091/2186] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:20 2022] DELETE /identity/v3/users/48f404324e17499da2b29cc8804b6eb7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:20.665140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:20.666725 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:20.682618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d502f-67c2-4686-9413-fe391ca61d3a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:20.696344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:20.746085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:20.746677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:20.746950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:20.747249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] PATH_INFO: `/v3/projects/913664a154df4f33bc5ada5b54298a50` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:20.748435 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913664a154df4f33bc5ada5b54298a50)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:20.750614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:20.798838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913664a154df4f33bc5ada5b54298a50', 'request_id': 'req-9e61f3b7-b760-4e20-821b-8d686d81a957'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.807001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913664a154df4f33bc5ada5b54298a50', 'request_id': 'req-9e61f3b7-b760-4e20-821b-8d686d81a957'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.813231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913664a154df4f33bc5ada5b54298a50', 'request_id': 'req-9e61f3b7-b760-4e20-821b-8d686d81a957'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.823943 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] The token cache is being invalidate because project 913664a154df4f33bc5ada5b54298a50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:20.824229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.824897 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.826500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1096/2187] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:20 2022] DELETE /identity/v3/projects/913664a154df4f33bc5ada5b54298a50 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:20.830602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:20.832282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:20.845833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d84bd-2455-4e45-8fcf-649280f673dc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:20.906451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:20.907079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:20.907358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:20.907627 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] PATH_INFO: `/v3/users/e2ba197d43ee48289b4b53c353ba935a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:20.908782 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ba197d43ee48289b4b53c353ba935a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:20.910737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:20.962568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ba197d43ee48289b4b53c353ba935a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.963235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ba197d43ee48289b4b53c353ba935a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.975324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ba197d43ee48289b4b53c353ba935a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.982023 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ba197d43ee48289b4b53c353ba935a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.982749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0c5a4f48-06ca-4d07-932f-e543d9d9ccf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ba197d43ee48289b4b53c353ba935a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:20.996999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1092/2188] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:20 2022] DELETE /identity/v3/users/e2ba197d43ee48289b4b53c353ba935a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:21.272407 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:21.273911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:21.287186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61f3b7-b760-4e20-821b-8d686d81a957 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:21.347034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:21.347655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.347930 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.348193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] PATH_INFO: `/v3/projects/331ed54c9e064ae4a9c09f8a78e3d799` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.349302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=331ed54c9e064ae4a9c09f8a78e3d799)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:21.351342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:21.398614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '331ed54c9e064ae4a9c09f8a78e3d799', 'request_id': 'req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:21.400247 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:21.408245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '331ed54c9e064ae4a9c09f8a78e3d799', 'request_id': 'req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:21.415995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '331ed54c9e064ae4a9c09f8a78e3d799', 'request_id': 'req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:21.426674 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] The token cache is being invalidate because project 331ed54c9e064ae4a9c09f8a78e3d799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:21.426955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:21.427606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d1d5d1-bf88-4fa8-bdb7-eeb6b7d17224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:21.429094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1097/2189] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:21 2022] DELETE /identity/v3/projects/331ed54c9e064ae4a9c09f8a78e3d799 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:21.435161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5bf36-2e4e-44ae-acff-c5e60fa17dd2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.435441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5bf36-2e4e-44ae-acff-c5e60fa17dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.435692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5bf36-2e4e-44ae-acff-c5e60fa17dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.456174 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4ea5bf36-2e4e-44ae-acff-c5e60fa17dd2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:21.457073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4ea5bf36-2e4e-44ae-acff-c5e60fa17dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:21.534177 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1093/2190] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:21.536166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b35414-f5b5-40a9-ad8d-d88baefc3a5a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.536457 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b35414-f5b5-40a9-ad8d-d88baefc3a5a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.536732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b35414-f5b5-40a9-ad8d-d88baefc3a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.559252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-49b35414-f5b5-40a9-ad8d-d88baefc3a5a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:21.560438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49b35414-f5b5-40a9-ad8d-d88baefc3a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:21.568914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1098/2191] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:21.573099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5bf36-2e4e-44ae-acff-c5e60fa17dd2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:21.583184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5bf36-2e4e-44ae-acff-c5e60fa17dd2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:21.588197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:21.588795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.589072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.589338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.590007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:21.590342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:21.591919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:21.594515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:21.606009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1094/2192] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:21.611351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b35414-f5b5-40a9-ad8d-d88baefc3a5a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:21.620927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b35414-f5b5-40a9-ad8d-d88baefc3a5a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:21.626226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94067c-df9f-4590-8322-acd12596b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:21.626831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94067c-df9f-4590-8322-acd12596b277 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.627109 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94067c-df9f-4590-8322-acd12596b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.627379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94067c-df9f-4590-8322-acd12596b277 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.628010 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94067c-df9f-4590-8322-acd12596b277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:21.630118 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94067c-df9f-4590-8322-acd12596b277 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:21.671398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1099/2193] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:21 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:21.672744 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:21.686197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c0ad3-b0ea-4f4f-8b6c-bd3345459498 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:21.748960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21ba4e-b246-4ad4-b0cb-107fb26ca155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:21.749561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21ba4e-b246-4ad4-b0cb-107fb26ca155 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.749837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21ba4e-b246-4ad4-b0cb-107fb26ca155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.750116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21ba4e-b246-4ad4-b0cb-107fb26ca155 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.750778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21ba4e-b246-4ad4-b0cb-107fb26ca155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:21.752802 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21ba4e-b246-4ad4-b0cb-107fb26ca155 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:21.775544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1095/2194] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:21 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:21.779506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94067c-df9f-4590-8322-acd12596b277 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:21.791367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94067c-df9f-4590-8322-acd12596b277 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:21.795987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4967c-5560-4e0d-8fc1-adefc695bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:21.796586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4967c-5560-4e0d-8fc1-adefc695bb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.796869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4967c-5560-4e0d-8fc1-adefc695bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.797143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4967c-5560-4e0d-8fc1-adefc695bb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.797784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4967c-5560-4e0d-8fc1-adefc695bb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:21.799487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4967c-5560-4e0d-8fc1-adefc695bb6c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:21.810666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1100/2195] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:37:21.814946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21ba4e-b246-4ad4-b0cb-107fb26ca155 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:21.825191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21ba4e-b246-4ad4-b0cb-107fb26ca155 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:21.829784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf0a93-effa-40f0-a880-7d468344ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:21.830414 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf0a93-effa-40f0-a880-7d468344ebe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.830696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf0a93-effa-40f0-a880-7d468344ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.831265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf0a93-effa-40f0-a880-7d468344ebe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.831592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf0a93-effa-40f0-a880-7d468344ebe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:21.833193 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf0a93-effa-40f0-a880-7d468344ebe5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:21.843779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1096/2196] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:37:21.848205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4967c-5560-4e0d-8fc1-adefc695bb6c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:21.858862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4967c-5560-4e0d-8fc1-adefc695bb6c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:21.863603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771fa94-6b51-4a1c-af62-f6198af756e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:21.864254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771fa94-6b51-4a1c-af62-f6198af756e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.864533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771fa94-6b51-4a1c-af62-f6198af756e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.864807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771fa94-6b51-4a1c-af62-f6198af756e5 admin admin] PATH_INFO: `/v3/projects/129c7f7ce032438a8c5588a4a7001f68/users/8349175bcce044839c5651cb77792237/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.880031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771fa94-6b51-4a1c-af62-f6198af756e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129c7f7ce032438a8c5588a4a7001f68, user_id=8349175bcce044839c5651cb77792237, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:21.889169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771fa94-6b51-4a1c-af62-f6198af756e5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:21.903951 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1101/2197] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:21 2022] PUT /identity/v3/projects/129c7f7ce032438a8c5588a4a7001f68/users/8349175bcce044839c5651cb77792237/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:21.916671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9afe48-625c-4ff6-8f86-e5b17dd6ef3a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:21.916959 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9afe48-625c-4ff6-8f86-e5b17dd6ef3a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:21.917232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9afe48-625c-4ff6-8f86-e5b17dd6ef3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:21.950463 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3c9afe48-625c-4ff6-8f86-e5b17dd6ef3a None None] MFA Rules not processed for user `8349175bcce044839c5651cb77792237`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:21.952083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3c9afe48-625c-4ff6-8f86-e5b17dd6ef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:22.020603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1097/2198] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:22.025343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771fa94-6b51-4a1c-af62-f6198af756e5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:22.087088 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771fa94-6b51-4a1c-af62-f6198af756e5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:22.091688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e64db4-54a7-4a60-bfb8-668f9d4428fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:22.092294 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e64db4-54a7-4a60-bfb8-668f9d4428fb admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:22.092568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e64db4-54a7-4a60-bfb8-668f9d4428fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:22.092837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e64db4-54a7-4a60-bfb8-668f9d4428fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:22.093474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e64db4-54a7-4a60-bfb8-668f9d4428fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:22.095443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e64db4-54a7-4a60-bfb8-668f9d4428fb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:22.129981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1102/2199] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:22 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:37:22.131779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9afe48-625c-4ff6-8f86-e5b17dd6ef3a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:22.143229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9afe48-625c-4ff6-8f86-e5b17dd6ef3a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:22.202242 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e350-296c-4c64-b2bd-f69640af3649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:22.202854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e350-296c-4c64-b2bd-f69640af3649 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:22.203134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e350-296c-4c64-b2bd-f69640af3649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:22.203407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e350-296c-4c64-b2bd-f69640af3649 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:22.204030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971e350-296c-4c64-b2bd-f69640af3649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:22.205935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971e350-296c-4c64-b2bd-f69640af3649 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:22.216448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:22.231257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1098/2200] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:22 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:22.235346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e64db4-54a7-4a60-bfb8-668f9d4428fb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:22.245607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e64db4-54a7-4a60-bfb8-668f9d4428fb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:22.250312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617386-d106-4c92-8ef5-05a36d58830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:22.250904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617386-d106-4c92-8ef5-05a36d58830e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:22.251181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617386-d106-4c92-8ef5-05a36d58830e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:22.251451 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617386-d106-4c92-8ef5-05a36d58830e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:22.252067 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0617386-d106-4c92-8ef5-05a36d58830e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:22.253683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0617386-d106-4c92-8ef5-05a36d58830e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:22.264499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1103/2201] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:22.268886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e350-296c-4c64-b2bd-f69640af3649 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:22.279444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e350-296c-4c64-b2bd-f69640af3649 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:22.284240 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef1308-b5ff-4ca2-8939-28ae5a4b4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:22.284887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef1308-b5ff-4ca2-8939-28ae5a4b4310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:22.285165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef1308-b5ff-4ca2-8939-28ae5a4b4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:22.285434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef1308-b5ff-4ca2-8939-28ae5a4b4310 admin admin] PATH_INFO: `/v3/projects/951afe7bcdbd487eacff8023ebba98a6/users/4b986cdb9f244e78b81242f4fc72cd58/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:22.301258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef1308-b5ff-4ca2-8939-28ae5a4b4310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=951afe7bcdbd487eacff8023ebba98a6, user_id=4b986cdb9f244e78b81242f4fc72cd58, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:22.304057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef1308-b5ff-4ca2-8939-28ae5a4b4310 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:22.318361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1099/2202] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:22 2022] PUT /identity/v3/projects/951afe7bcdbd487eacff8023ebba98a6/users/4b986cdb9f244e78b81242f4fc72cd58/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:22.322999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617386-d106-4c92-8ef5-05a36d58830e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:22.333491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617386-d106-4c92-8ef5-05a36d58830e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:22.390908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:22.405737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788ded2-c29e-4539-81fe-2fcfcc12405a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:22.406385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788ded2-c29e-4539-81fe-2fcfcc12405a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:22.406667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788ded2-c29e-4539-81fe-2fcfcc12405a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:22.406941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788ded2-c29e-4539-81fe-2fcfcc12405a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:22.407585 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788ded2-c29e-4539-81fe-2fcfcc12405a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:22.409223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788ded2-c29e-4539-81fe-2fcfcc12405a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:22.419924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1104/2203] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:22.424408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef1308-b5ff-4ca2-8939-28ae5a4b4310 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:22.436799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef1308-b5ff-4ca2-8939-28ae5a4b4310 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:22.443101 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:22.444332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:22.444601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:22.444867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] PATH_INFO: `/v3/projects/951afe7bcdbd487eacff8023ebba98a6/users/4b986cdb9f244e78b81242f4fc72cd58/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:22.447085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=951afe7bcdbd487eacff8023ebba98a6, user_id=4b986cdb9f244e78b81242f4fc72cd58, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:22.449837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:22.462169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1100/2204] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:22 2022] PUT /identity/v3/projects/951afe7bcdbd487eacff8023ebba98a6/users/4b986cdb9f244e78b81242f4fc72cd58/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:22.477310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2dd7f-d2f0-41ef-adb6-9019b7a6b468 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:22.477612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2dd7f-d2f0-41ef-adb6-9019b7a6b468 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:22.478001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2dd7f-d2f0-41ef-adb6-9019b7a6b468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:22.511774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5db2dd7f-d2f0-41ef-adb6-9019b7a6b468 None None] MFA Rules not processed for user `4b986cdb9f244e78b81242f4fc72cd58`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:22.512980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5db2dd7f-d2f0-41ef-adb6-9019b7a6b468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:22.587360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1105/2205] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:22.599405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:22.601505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:22.665596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78656546-608f-404f-aa0e-6a04080e01ce admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:22.669700 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974a9f1-d89e-425d-a242-bd98522fabf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:22.670316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974a9f1-d89e-425d-a242-bd98522fabf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:22.670606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974a9f1-d89e-425d-a242-bd98522fabf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:22.670865 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974a9f1-d89e-425d-a242-bd98522fabf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:22.681324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974a9f1-d89e-425d-a242-bd98522fabf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:22.682817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974a9f1-d89e-425d-a242-bd98522fabf2 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:22.692644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1101/2206] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:23.100864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2dd7f-d2f0-41ef-adb6-9019b7a6b468 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:23.168553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2dd7f-d2f0-41ef-adb6-9019b7a6b468 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:23.173314 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:23.173940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:23.174261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:23.174533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] PATH_INFO: `/v3/users/8349175bcce044839c5651cb77792237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:23.175794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8349175bcce044839c5651cb77792237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:23.177851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:23.235552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8349175bcce044839c5651cb77792237'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.236328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8349175bcce044839c5651cb77792237'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.250289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8349175bcce044839c5651cb77792237'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.255811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:23.263396 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8349175bcce044839c5651cb77792237'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.274124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8349175bcce044839c5651cb77792237'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.296829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1106/2207] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:23 2022] DELETE /identity/v3/users/8349175bcce044839c5651cb77792237 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:23.301608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46479-1691-42fe-aaf3-e2c53f4c071e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:23.302053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46479-1691-42fe-aaf3-e2c53f4c071e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:23.302356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46479-1691-42fe-aaf3-e2c53f4c071e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:23.324957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0cc46479-1691-42fe-aaf3-e2c53f4c071e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:23.326245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0cc46479-1691-42fe-aaf3-e2c53f4c071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:23.394202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1102/2208] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:23.407420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:23.408817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:23.469491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac974880-fc2b-4b4e-b68e-f27bbf285ecb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:23.474468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806c20c-1959-4650-a55a-99b8c92e1350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:23.475364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806c20c-1959-4650-a55a-99b8c92e1350 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:23.475774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806c20c-1959-4650-a55a-99b8c92e1350 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:23.476052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806c20c-1959-4650-a55a-99b8c92e1350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:23.486261 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806c20c-1959-4650-a55a-99b8c92e1350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:23.487696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806c20c-1959-4650-a55a-99b8c92e1350 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:23.497861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1107/2209] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:37:23.793075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc46479-1691-42fe-aaf3-e2c53f4c071e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:23.794537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0cc46479-1691-42fe-aaf3-e2c53f4c071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:23.806966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc46479-1691-42fe-aaf3-e2c53f4c071e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:23.811499 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:23.812102 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:23.812416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:23.812666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] PATH_INFO: `/v3/projects/129c7f7ce032438a8c5588a4a7001f68` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:23.813859 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] RBAC: Authorizing `identity:delete_project(project_id=129c7f7ce032438a8c5588a4a7001f68)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:23.815997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:23.866557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129c7f7ce032438a8c5588a4a7001f68', 'request_id': 'req-f567539d-3762-4102-91f4-fb435896eefe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.873117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129c7f7ce032438a8c5588a4a7001f68', 'request_id': 'req-f567539d-3762-4102-91f4-fb435896eefe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.874781 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:23.882284 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129c7f7ce032438a8c5588a4a7001f68', 'request_id': 'req-f567539d-3762-4102-91f4-fb435896eefe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.894749 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] The token cache is being invalidate because project 129c7f7ce032438a8c5588a4a7001f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:23.895119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.895834 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:23.898254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1103/2210] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:23 2022] DELETE /identity/v3/projects/129c7f7ce032438a8c5588a4a7001f68 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:23.902398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806c20c-1959-4650-a55a-99b8c92e1350 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:23.914114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806c20c-1959-4650-a55a-99b8c92e1350 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:23.975998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:23.976621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:23.976906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:23.977197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] PATH_INFO: `/v3/users/4b986cdb9f244e78b81242f4fc72cd58` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:23.978459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b986cdb9f244e78b81242f4fc72cd58)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:23.980671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:24.032679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b986cdb9f244e78b81242f4fc72cd58'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.033351 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b986cdb9f244e78b81242f4fc72cd58'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.046766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b986cdb9f244e78b81242f4fc72cd58'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.055212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b986cdb9f244e78b81242f4fc72cd58'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.056020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-227acfce-1df9-49a0-8a62-e1bd60d551a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b986cdb9f244e78b81242f4fc72cd58'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.072552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1108/2211] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:23 2022] DELETE /identity/v3/users/4b986cdb9f244e78b81242f4fc72cd58 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:24.356226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:24.357707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:24.375757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567539d-3762-4102-91f4-fb435896eefe admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:24.449803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:24.450483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:24.450762 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:24.451034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] PATH_INFO: `/v3/projects/951afe7bcdbd487eacff8023ebba98a6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:24.452180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=951afe7bcdbd487eacff8023ebba98a6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:24.454502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:24.507534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951afe7bcdbd487eacff8023ebba98a6', 'request_id': 'req-1e1d2603-422d-42f8-be45-412090b197f0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.514451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951afe7bcdbd487eacff8023ebba98a6', 'request_id': 'req-1e1d2603-422d-42f8-be45-412090b197f0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.523287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951afe7bcdbd487eacff8023ebba98a6', 'request_id': 'req-1e1d2603-422d-42f8-be45-412090b197f0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.534438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] The token cache is being invalidate because project 951afe7bcdbd487eacff8023ebba98a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:24.534848 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.535579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e1d2603-422d-42f8-be45-412090b197f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:24.537378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1104/2212] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:24 2022] DELETE /identity/v3/projects/951afe7bcdbd487eacff8023ebba98a6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:24.543539 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae74b-b5e6-4f18-a046-3cc281a35a74 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:24.545018 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae74b-b5e6-4f18-a046-3cc281a35a74 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:24.545374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaae74b-b5e6-4f18-a046-3cc281a35a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:24.567264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6aaae74b-b5e6-4f18-a046-3cc281a35a74 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:24.568506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6aaae74b-b5e6-4f18-a046-3cc281a35a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:24.575714 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:24.653129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1109/2213] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:24.662654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b3a73-e569-4c0c-a824-467c285dc268 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:24.663019 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b3a73-e569-4c0c-a824-467c285dc268 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:24.663335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b3a73-e569-4c0c-a824-467c285dc268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:24.685108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-373b3a73-e569-4c0c-a824-467c285dc268 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:24.686430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-373b3a73-e569-4c0c-a824-467c285dc268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:24.696849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1105/2214] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:24.701319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaae74b-b5e6-4f18-a046-3cc281a35a74 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:24.713246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaae74b-b5e6-4f18-a046-3cc281a35a74 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:24.717882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:24.718621 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:24.718885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:24.719232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:24.719950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:24.720259 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:24.722383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:24.738136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1110/2215] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:24.740863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b3a73-e569-4c0c-a824-467c285dc268 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:24.752941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b3a73-e569-4c0c-a824-467c285dc268 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:24.758976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8db030-6acc-4b68-81ee-8d2713d1445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:24.759610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8db030-6acc-4b68-81ee-8d2713d1445a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:24.759893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8db030-6acc-4b68-81ee-8d2713d1445a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:24.760314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8db030-6acc-4b68-81ee-8d2713d1445a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:24.760814 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8db030-6acc-4b68-81ee-8d2713d1445a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:24.762760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8db030-6acc-4b68-81ee-8d2713d1445a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:24.801603 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1106/2216] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:24 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:24.802657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:24.813254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543d25b-20ab-459c-a07a-a8a2930435c3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:24.877186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb320f43-86d7-43eb-85b4-1364db056817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:24.877866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb320f43-86d7-43eb-85b4-1364db056817 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:24.878142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb320f43-86d7-43eb-85b4-1364db056817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:24.878412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb320f43-86d7-43eb-85b4-1364db056817 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:24.879049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb320f43-86d7-43eb-85b4-1364db056817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:24.880964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb320f43-86d7-43eb-85b4-1364db056817 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:24.910500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1111/2217] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:24 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:37:24.915941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8db030-6acc-4b68-81ee-8d2713d1445a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:24.927605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8db030-6acc-4b68-81ee-8d2713d1445a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:24.933703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cefc29-50ee-4e40-ba38-1747c75a1c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:24.934561 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefc29-50ee-4e40-ba38-1747c75a1c07 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:24.934875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefc29-50ee-4e40-ba38-1747c75a1c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:24.935153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefc29-50ee-4e40-ba38-1747c75a1c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:24.936003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cefc29-50ee-4e40-ba38-1747c75a1c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:24.937819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cefc29-50ee-4e40-ba38-1747c75a1c07 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:24.953871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1107/2218] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:24.955170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb320f43-86d7-43eb-85b4-1364db056817 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:24.966004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb320f43-86d7-43eb-85b4-1364db056817 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:24.971991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974b3bf-d234-43a1-a57f-9a8ae2dc8199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:24.972696 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b3bf-d234-43a1-a57f-9a8ae2dc8199 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:24.973016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b3bf-d234-43a1-a57f-9a8ae2dc8199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:24.973354 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b3bf-d234-43a1-a57f-9a8ae2dc8199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:24.974200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974b3bf-d234-43a1-a57f-9a8ae2dc8199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:24.976432 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974b3bf-d234-43a1-a57f-9a8ae2dc8199 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:24.987855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1112/2219] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:24.992478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cefc29-50ee-4e40-ba38-1747c75a1c07 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.003152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cefc29-50ee-4e40-ba38-1747c75a1c07 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.008795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ad9a8-206b-4956-ab12-bd7730700afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.009456 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ad9a8-206b-4956-ab12-bd7730700afa admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.009786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ad9a8-206b-4956-ab12-bd7730700afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.010058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ad9a8-206b-4956-ab12-bd7730700afa admin admin] PATH_INFO: `/v3/projects/592f1d61ef0e4e3d82a27f340b3506a1/users/3e8da3492b794278ab6f9294e4cf416b/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.025853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ad9a8-206b-4956-ab12-bd7730700afa admin admin] RBAC: Authorizing `identity:create_grant(project_id=592f1d61ef0e4e3d82a27f340b3506a1, user_id=3e8da3492b794278ab6f9294e4cf416b, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.028528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ad9a8-206b-4956-ab12-bd7730700afa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.043890 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1108/2220] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:24 2022] PUT /identity/v3/projects/592f1d61ef0e4e3d82a27f340b3506a1/users/3e8da3492b794278ab6f9294e4cf416b/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:25.054582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786b10-77c9-4e21-b81b-2f8023b32347 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.054832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786b10-77c9-4e21-b81b-2f8023b32347 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.055068 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786b10-77c9-4e21-b81b-2f8023b32347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.083639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-89786b10-77c9-4e21-b81b-2f8023b32347 None None] MFA Rules not processed for user `3e8da3492b794278ab6f9294e4cf416b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:25.084901 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-89786b10-77c9-4e21-b81b-2f8023b32347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:25.154290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1113/2221] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:25.159276 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ad9a8-206b-4956-ab12-bd7730700afa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.170301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:25.225660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ad9a8-206b-4956-ab12-bd7730700afa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.230350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255740e-a215-4f6d-b4bf-4fcfac4d3724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.230986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255740e-a215-4f6d-b4bf-4fcfac4d3724 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.231265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255740e-a215-4f6d-b4bf-4fcfac4d3724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.231536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255740e-a215-4f6d-b4bf-4fcfac4d3724 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.232169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b255740e-a215-4f6d-b4bf-4fcfac4d3724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.234146 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b255740e-a215-4f6d-b4bf-4fcfac4d3724 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.265831 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1109/2222] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:25 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:25.270694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89786b10-77c9-4e21-b81b-2f8023b32347 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.286950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89786b10-77c9-4e21-b81b-2f8023b32347 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.291579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:25.350928 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a76091-19c7-466a-b32d-022189e69d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.351617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a76091-19c7-466a-b32d-022189e69d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.351898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a76091-19c7-466a-b32d-022189e69d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.352167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a76091-19c7-466a-b32d-022189e69d8c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.352979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a76091-19c7-466a-b32d-022189e69d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.355149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a76091-19c7-466a-b32d-022189e69d8c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.379290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1114/2223] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:25 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:37:25.384184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255740e-a215-4f6d-b4bf-4fcfac4d3724 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.395015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255740e-a215-4f6d-b4bf-4fcfac4d3724 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.400340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2c160-f610-4e9f-ad03-1265e0e93875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.400929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2c160-f610-4e9f-ad03-1265e0e93875 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.401200 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2c160-f610-4e9f-ad03-1265e0e93875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.401466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2c160-f610-4e9f-ad03-1265e0e93875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.402112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2c160-f610-4e9f-ad03-1265e0e93875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.403721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2c160-f610-4e9f-ad03-1265e0e93875 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.414966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1110/2224] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:25.419210 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a76091-19c7-466a-b32d-022189e69d8c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.430295 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a76091-19c7-466a-b32d-022189e69d8c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.435834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e503c3-696a-408b-9bfd-9e24cbec8304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.436444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e503c3-696a-408b-9bfd-9e24cbec8304 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.436718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e503c3-696a-408b-9bfd-9e24cbec8304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.436994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e503c3-696a-408b-9bfd-9e24cbec8304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.437623 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e503c3-696a-408b-9bfd-9e24cbec8304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.439275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e503c3-696a-408b-9bfd-9e24cbec8304 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.454692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1115/2225] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:37:25.456066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2c160-f610-4e9f-ad03-1265e0e93875 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.468194 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2c160-f610-4e9f-ad03-1265e0e93875 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.473938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae835f9a-2706-4be4-8b28-b7d67a600709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.474639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae835f9a-2706-4be4-8b28-b7d67a600709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.475069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae835f9a-2706-4be4-8b28-b7d67a600709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.475419 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae835f9a-2706-4be4-8b28-b7d67a600709 admin admin] PATH_INFO: `/v3/projects/9de44a38a06b403c8440daa8767a17ac/users/97b025a754a5473b89e31691b765ae3e/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.492496 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae835f9a-2706-4be4-8b28-b7d67a600709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de44a38a06b403c8440daa8767a17ac, user_id=97b025a754a5473b89e31691b765ae3e, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.495324 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae835f9a-2706-4be4-8b28-b7d67a600709 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.510529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1111/2226] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:25 2022] PUT /identity/v3/projects/9de44a38a06b403c8440daa8767a17ac/users/97b025a754a5473b89e31691b765ae3e/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:25.520965 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a225f38-e593-433b-8478-66837a7d61b0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.521249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a225f38-e593-433b-8478-66837a7d61b0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.521518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a225f38-e593-433b-8478-66837a7d61b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.553408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1a225f38-e593-433b-8478-66837a7d61b0 None None] MFA Rules not processed for user `97b025a754a5473b89e31691b765ae3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:25.554757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1a225f38-e593-433b-8478-66837a7d61b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:25.631226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1116/2227] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:25.635864 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae835f9a-2706-4be4-8b28-b7d67a600709 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.702608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae835f9a-2706-4be4-8b28-b7d67a600709 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.707588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883999f4-4088-4024-97dc-63b78a1a03b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.708132 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883999f4-4088-4024-97dc-63b78a1a03b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.708409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883999f4-4088-4024-97dc-63b78a1a03b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.708681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883999f4-4088-4024-97dc-63b78a1a03b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.709339 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883999f4-4088-4024-97dc-63b78a1a03b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.713621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883999f4-4088-4024-97dc-63b78a1a03b0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.745854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1112/2228] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:25 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:37:25.750317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a225f38-e593-433b-8478-66837a7d61b0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.762487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a225f38-e593-433b-8478-66837a7d61b0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.823991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee2cac-bdca-4c51-86b1-7ca546e6e1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.824703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee2cac-bdca-4c51-86b1-7ca546e6e1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.824869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee2cac-bdca-4c51-86b1-7ca546e6e1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.825169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee2cac-bdca-4c51-86b1-7ca546e6e1c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.825808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee2cac-bdca-4c51-86b1-7ca546e6e1c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.827759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee2cac-bdca-4c51-86b1-7ca546e6e1c4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.852231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1117/2229] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:25 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:25.856769 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883999f4-4088-4024-97dc-63b78a1a03b0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.868957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883999f4-4088-4024-97dc-63b78a1a03b0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.873747 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bb2cd-eee9-4ae8-806c-344b713c8469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.874386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bb2cd-eee9-4ae8-806c-344b713c8469 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.874665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bb2cd-eee9-4ae8-806c-344b713c8469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.874936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bb2cd-eee9-4ae8-806c-344b713c8469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.875610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4bb2cd-eee9-4ae8-806c-344b713c8469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.877269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4bb2cd-eee9-4ae8-806c-344b713c8469 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.895180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1113/2230] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:25.897445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee2cac-bdca-4c51-86b1-7ca546e6e1c4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.904248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:25.908398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee2cac-bdca-4c51-86b1-7ca546e6e1c4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.915107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8094e-2319-46ac-a463-0774278b06d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:25.915790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8094e-2319-46ac-a463-0774278b06d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:25.916053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8094e-2319-46ac-a463-0774278b06d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:25.916316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8094e-2319-46ac-a463-0774278b06d6 admin admin] PATH_INFO: `/v3/projects/9ca32c95d7284827ac2d7862c8572995/users/1fd92e45d51d4595b51b052ec12ac9d7/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:25.932897 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8094e-2319-46ac-a463-0774278b06d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca32c95d7284827ac2d7862c8572995, user_id=1fd92e45d51d4595b51b052ec12ac9d7, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:25.935599 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8094e-2319-46ac-a463-0774278b06d6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:25.949653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1118/2231] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:25 2022] PUT /identity/v3/projects/9ca32c95d7284827ac2d7862c8572995/users/1fd92e45d51d4595b51b052ec12ac9d7/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:25.953694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bb2cd-eee9-4ae8-806c-344b713c8469 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:25.966068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bb2cd-eee9-4ae8-806c-344b713c8469 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:25.999417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:26.027858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641561d-7e2f-4f67-a5d1-3a33a2e7f454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:26.028581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641561d-7e2f-4f67-a5d1-3a33a2e7f454 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:26.028861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641561d-7e2f-4f67-a5d1-3a33a2e7f454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:26.029197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641561d-7e2f-4f67-a5d1-3a33a2e7f454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:26.029937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641561d-7e2f-4f67-a5d1-3a33a2e7f454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:26.031930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641561d-7e2f-4f67-a5d1-3a33a2e7f454 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:26.044301 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1114/2232] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:26.049190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8094e-2319-46ac-a463-0774278b06d6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:26.060412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8094e-2319-46ac-a463-0774278b06d6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:26.065773 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:26.066470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:26.066781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:26.067055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] PATH_INFO: `/v3/projects/9ca32c95d7284827ac2d7862c8572995/users/1fd92e45d51d4595b51b052ec12ac9d7/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:26.069212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca32c95d7284827ac2d7862c8572995, user_id=1fd92e45d51d4595b51b052ec12ac9d7, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:26.071946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:26.083332 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1119/2233] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:26 2022] PUT /identity/v3/projects/9ca32c95d7284827ac2d7862c8572995/users/1fd92e45d51d4595b51b052ec12ac9d7/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:26.095392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5a36-1550-4fab-be9b-5df6d3652f6f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:26.095660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5a36-1550-4fab-be9b-5df6d3652f6f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:26.095954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5a36-1550-4fab-be9b-5df6d3652f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:26.126872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-272f5a36-1550-4fab-be9b-5df6d3652f6f None None] MFA Rules not processed for user `1fd92e45d51d4595b51b052ec12ac9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:26.128104 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-272f5a36-1550-4fab-be9b-5df6d3652f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:26.202514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1115/2234] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:26.214085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:26.215879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:26.277801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86366d-45e7-42ec-920b-454ddf836181 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:26.281769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddb740-951a-459b-8184-aead397e6979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:26.282374 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddb740-951a-459b-8184-aead397e6979 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:26.282649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddb740-951a-459b-8184-aead397e6979 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:26.282917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddb740-951a-459b-8184-aead397e6979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:26.343037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddb740-951a-459b-8184-aead397e6979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:26.344584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddb740-951a-459b-8184-aead397e6979 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:26.354870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1120/2235] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:27.894384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f5a36-1550-4fab-be9b-5df6d3652f6f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:27.895901 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-272f5a36-1550-4fab-be9b-5df6d3652f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:27.907999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f5a36-1550-4fab-be9b-5df6d3652f6f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:27.912213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33601-28d6-409b-92d4-e570f7daaae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:27.912866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33601-28d6-409b-92d4-e570f7daaae5 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:27.913144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33601-28d6-409b-92d4-e570f7daaae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:27.913416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33601-28d6-409b-92d4-e570f7daaae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:27.979010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe33601-28d6-409b-92d4-e570f7daaae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:27.980493 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe33601-28d6-409b-92d4-e570f7daaae5 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:27.989800 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1116/2236] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:28.572641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddb740-951a-459b-8184-aead397e6979 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:28.582974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddb740-951a-459b-8184-aead397e6979 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:28.588097 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec180e-dbc3-4b1e-8a63-108d00168ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:28.588713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec180e-dbc3-4b1e-8a63-108d00168ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:28.588984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec180e-dbc3-4b1e-8a63-108d00168ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:28.589248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec180e-dbc3-4b1e-8a63-108d00168ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:28.599409 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec180e-dbc3-4b1e-8a63-108d00168ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:28.600872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec180e-dbc3-4b1e-8a63-108d00168ff2 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:28.610871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1121/2237] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:33.744054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33601-28d6-409b-92d4-e570f7daaae5 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:33.797108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:33.809021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33601-28d6-409b-92d4-e570f7daaae5 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:33.814168 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:33.814416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:33.814796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:33.815071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] PATH_INFO: `/v3/users/3e8da3492b794278ab6f9294e4cf416b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:33.816351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8da3492b794278ab6f9294e4cf416b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:33.818590 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:33.865521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8da3492b794278ab6f9294e4cf416b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:33.866256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8da3492b794278ab6f9294e4cf416b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:33.879003 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8da3492b794278ab6f9294e4cf416b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:33.885889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8da3492b794278ab6f9294e4cf416b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:33.886566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8da3492b794278ab6f9294e4cf416b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:33.901375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1117/2238] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:33 2022] DELETE /identity/v3/users/3e8da3492b794278ab6f9294e4cf416b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:33.906003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1dad6-e0c7-4add-b515-7a2025fd087b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:33.906517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1dad6-e0c7-4add-b515-7a2025fd087b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:33.906695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1dad6-e0c7-4add-b515-7a2025fd087b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:33.927443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5fc1dad6-e0c7-4add-b515-7a2025fd087b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:33.928624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5fc1dad6-e0c7-4add-b515-7a2025fd087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:33.996479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1122/2239] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:34.008625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:34.010970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:34.075404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f1f68-df09-4d68-a02b-b69a530fd114 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:34.079421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288532-b1c9-46a4-8039-72cce7528453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:34.080012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288532-b1c9-46a4-8039-72cce7528453 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:34.080260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288532-b1c9-46a4-8039-72cce7528453 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:34.080550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288532-b1c9-46a4-8039-72cce7528453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:34.090246 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80288532-b1c9-46a4-8039-72cce7528453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:34.091657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80288532-b1c9-46a4-8039-72cce7528453 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:34.101336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1118/2240] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:34.389841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1dad6-e0c7-4add-b515-7a2025fd087b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:34.391391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5fc1dad6-e0c7-4add-b515-7a2025fd087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:34.405519 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1dad6-e0c7-4add-b515-7a2025fd087b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:34.412827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:34.413463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:34.413737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:34.414001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] PATH_INFO: `/v3/projects/592f1d61ef0e4e3d82a27f340b3506a1` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:34.415470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=592f1d61ef0e4e3d82a27f340b3506a1)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:34.417622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:34.420967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:34.473683 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592f1d61ef0e4e3d82a27f340b3506a1', 'request_id': 'req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.481625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592f1d61ef0e4e3d82a27f340b3506a1', 'request_id': 'req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.488179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592f1d61ef0e4e3d82a27f340b3506a1', 'request_id': 'req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.498994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] The token cache is being invalidate because project 592f1d61ef0e4e3d82a27f340b3506a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:34.499280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.499998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.501576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1123/2241] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:34 2022] DELETE /identity/v3/projects/592f1d61ef0e4e3d82a27f340b3506a1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:34.505920 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288532-b1c9-46a4-8039-72cce7528453 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:34.516018 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288532-b1c9-46a4-8039-72cce7528453 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:34.583053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:34.583669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:34.583948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:34.584219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] PATH_INFO: `/v3/users/97b025a754a5473b89e31691b765ae3e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:34.585378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b025a754a5473b89e31691b765ae3e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:34.587590 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:34.604759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:34.639617 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b025a754a5473b89e31691b765ae3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.640398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b025a754a5473b89e31691b765ae3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.652257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b025a754a5473b89e31691b765ae3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.659244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b025a754a5473b89e31691b765ae3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.659916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b025a754a5473b89e31691b765ae3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:34.674776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1119/2242] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:34 2022] DELETE /identity/v3/users/97b025a754a5473b89e31691b765ae3e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:34.962729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:34.964211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:34.978501 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a181b8-a4d5-414a-96ab-61d08d7a77f7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:35.043370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:35.044170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:35.044310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:35.044529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] PATH_INFO: `/v3/projects/9de44a38a06b403c8440daa8767a17ac` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:35.045641 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de44a38a06b403c8440daa8767a17ac)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:35.047565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:35.093515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de44a38a06b403c8440daa8767a17ac', 'request_id': 'req-a1b0314b-685f-4614-b4ea-f61c91dc686a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.101086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de44a38a06b403c8440daa8767a17ac', 'request_id': 'req-a1b0314b-685f-4614-b4ea-f61c91dc686a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.107433 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de44a38a06b403c8440daa8767a17ac', 'request_id': 'req-a1b0314b-685f-4614-b4ea-f61c91dc686a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.119058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] The token cache is being invalidate because project 9de44a38a06b403c8440daa8767a17ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:35.119361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.120012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.121708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1124/2243] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:34 2022] DELETE /identity/v3/projects/9de44a38a06b403c8440daa8767a17ac => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:35.125853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:35.127336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:35.140652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f70ff9-3fd7-40c0-8590-4620d0c3f5d4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:35.199086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:35.199715 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:35.199995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:35.200268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] PATH_INFO: `/v3/users/1fd92e45d51d4595b51b052ec12ac9d7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:35.201394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd92e45d51d4595b51b052ec12ac9d7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:35.203358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:35.252326 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd92e45d51d4595b51b052ec12ac9d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.253153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd92e45d51d4595b51b052ec12ac9d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.266248 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd92e45d51d4595b51b052ec12ac9d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.272827 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd92e45d51d4595b51b052ec12ac9d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.273470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed11c6c8-8797-4a74-9938-ad71ad61a8b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd92e45d51d4595b51b052ec12ac9d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.285714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1120/2244] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:35 2022] DELETE /identity/v3/users/1fd92e45d51d4595b51b052ec12ac9d7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:35.583039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:35.584491 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:35.604350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0314b-685f-4614-b4ea-f61c91dc686a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:35.624062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:35.665201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:35.666193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:35.666474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:35.666746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] PATH_INFO: `/v3/projects/9ca32c95d7284827ac2d7862c8572995` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:35.667889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca32c95d7284827ac2d7862c8572995)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:35.669791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:35.722189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca32c95d7284827ac2d7862c8572995', 'request_id': 'req-e63630b7-453d-49d8-9c71-03f38551a839'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.731910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca32c95d7284827ac2d7862c8572995', 'request_id': 'req-e63630b7-453d-49d8-9c71-03f38551a839'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.738971 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca32c95d7284827ac2d7862c8572995', 'request_id': 'req-e63630b7-453d-49d8-9c71-03f38551a839'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.750780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] The token cache is being invalidate because project 9ca32c95d7284827ac2d7862c8572995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:35.751070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.751715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e63630b7-453d-49d8-9c71-03f38551a839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:35.753237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1125/2245] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:35 2022] DELETE /identity/v3/projects/9ca32c95d7284827ac2d7862c8572995 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:35.759124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb87480-46dd-4b9c-9d36-4bf720800b3e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:35.759467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb87480-46dd-4b9c-9d36-4bf720800b3e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:35.759741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb87480-46dd-4b9c-9d36-4bf720800b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:35.779645 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-afb87480-46dd-4b9c-9d36-4bf720800b3e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:35.780820 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-afb87480-46dd-4b9c-9d36-4bf720800b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:35.789382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:35.849579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1121/2246] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:35.858899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dece26-5a21-4e75-9fe7-7f2c150c640e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:35.859236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dece26-5a21-4e75-9fe7-7f2c150c640e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:35.859576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dece26-5a21-4e75-9fe7-7f2c150c640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:35.883368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a2dece26-5a21-4e75-9fe7-7f2c150c640e None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:35.884594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a2dece26-5a21-4e75-9fe7-7f2c150c640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:35.894211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1126/2247] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:35.898696 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb87480-46dd-4b9c-9d36-4bf720800b3e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:35.909404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb87480-46dd-4b9c-9d36-4bf720800b3e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:35.914748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:35.915370 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:35.915986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:35.915986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:35.916535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:35.916832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:35.918450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:35.931942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1122/2248] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:35.936527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dece26-5a21-4e75-9fe7-7f2c150c640e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:35.945908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dece26-5a21-4e75-9fe7-7f2c150c640e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:35.951356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d66f9-2794-4acf-a8aa-7ccf93cee87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:35.951968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d66f9-2794-4acf-a8aa-7ccf93cee87f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:35.952218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d66f9-2794-4acf-a8aa-7ccf93cee87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:35.952486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d66f9-2794-4acf-a8aa-7ccf93cee87f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:35.953103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d66f9-2794-4acf-a8aa-7ccf93cee87f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:35.955008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d66f9-2794-4acf-a8aa-7ccf93cee87f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:35.984342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1127/2249] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:35 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:35.988579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:35.999015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee142120-9fb9-4621-854f-e1fde56cde0b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:36.055281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a87a-2079-44c0-a0a0-896c900f0139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:36.055865 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a87a-2079-44c0-a0a0-896c900f0139 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.056637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a87a-2079-44c0-a0a0-896c900f0139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.056856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a87a-2079-44c0-a0a0-896c900f0139 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.057729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3a87a-2079-44c0-a0a0-896c900f0139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:36.059654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3a87a-2079-44c0-a0a0-896c900f0139 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:36.082502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1123/2250] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:35 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:36.086812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d66f9-2794-4acf-a8aa-7ccf93cee87f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:36.097895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d66f9-2794-4acf-a8aa-7ccf93cee87f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:36.102552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c55a4c-a0f3-4c31-8782-d3e4d795d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:36.103203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55a4c-a0f3-4c31-8782-d3e4d795d8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.103476 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55a4c-a0f3-4c31-8782-d3e4d795d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.103755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55a4c-a0f3-4c31-8782-d3e4d795d8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.104423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c55a4c-a0f3-4c31-8782-d3e4d795d8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:36.106063 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c55a4c-a0f3-4c31-8782-d3e4d795d8e4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:36.116097 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1128/2251] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:36.120183 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a87a-2079-44c0-a0a0-896c900f0139 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:36.130350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a87a-2079-44c0-a0a0-896c900f0139 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:36.135849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d346164-0991-4653-a92f-c6abfdd3b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:36.136446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d346164-0991-4653-a92f-c6abfdd3b2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.136725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d346164-0991-4653-a92f-c6abfdd3b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.136994 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d346164-0991-4653-a92f-c6abfdd3b2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.137630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d346164-0991-4653-a92f-c6abfdd3b2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:36.139300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d346164-0991-4653-a92f-c6abfdd3b2b3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:36.149267 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1124/2252] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:36.153826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c55a4c-a0f3-4c31-8782-d3e4d795d8e4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:36.164139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c55a4c-a0f3-4c31-8782-d3e4d795d8e4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:36.169233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:36.169880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.170200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.170478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] PATH_INFO: `/v3/projects/d7cf1d0683bc47fba160946269925d48/users/285ce44565bf454f9e8258f48fc88f08/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.187310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7cf1d0683bc47fba160946269925d48, user_id=285ce44565bf454f9e8258f48fc88f08, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:36.190151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:36.205172 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1129/2253] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:36 2022] PUT /identity/v3/projects/d7cf1d0683bc47fba160946269925d48/users/285ce44565bf454f9e8258f48fc88f08/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:36.276544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeb462-a9c4-4aed-a9d8-82d01c039b76 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.276849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeb462-a9c4-4aed-a9d8-82d01c039b76 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.277092 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeb462-a9c4-4aed-a9d8-82d01c039b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.308977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3ceeb462-a9c4-4aed-a9d8-82d01c039b76 None None] MFA Rules not processed for user `285ce44565bf454f9e8258f48fc88f08`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:36.310338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ceeb462-a9c4-4aed-a9d8-82d01c039b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:36.389655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1125/2254] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:36.390848 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:36.392718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:36.456523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:36.461402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3010-e1c4-4cc0-a79b-5406fd33e1d1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:36.466586 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc426ad5-5d98-4806-9397-d9d482f01c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:36.467240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc426ad5-5d98-4806-9397-d9d482f01c34 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:36.467560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc426ad5-5d98-4806-9397-d9d482f01c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:36.467846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc426ad5-5d98-4806-9397-d9d482f01c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:36.478736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc426ad5-5d98-4806-9397-d9d482f01c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:36.480271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc426ad5-5d98-4806-9397-d9d482f01c34 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:36.490032 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1130/2255] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:40.179974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceeb462-a9c4-4aed-a9d8-82d01c039b76 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:40.195287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:40.253887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceeb462-a9c4-4aed-a9d8-82d01c039b76 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:40.258539 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:40.259161 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:40.259437 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:40.259809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] PATH_INFO: `/v3/users/285ce44565bf454f9e8258f48fc88f08` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:40.261550 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=285ce44565bf454f9e8258f48fc88f08)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:40.263637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:40.312628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '285ce44565bf454f9e8258f48fc88f08'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.313311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '285ce44565bf454f9e8258f48fc88f08'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.327014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '285ce44565bf454f9e8258f48fc88f08'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.334310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '285ce44565bf454f9e8258f48fc88f08'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.335017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '285ce44565bf454f9e8258f48fc88f08'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.349869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1126/2256] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:40 2022] DELETE /identity/v3/users/285ce44565bf454f9e8258f48fc88f08 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:40.354909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be06f-ebcb-464d-89d0-74486abaeb46 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:40.355446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be06f-ebcb-464d-89d0-74486abaeb46 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:40.355446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be06f-ebcb-464d-89d0-74486abaeb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:40.376479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-632be06f-ebcb-464d-89d0-74486abaeb46 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:40.377675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-632be06f-ebcb-464d-89d0-74486abaeb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:40.440450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1131/2257] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:40.450728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:40.452313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:40.513243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60d696-ae27-4feb-9b4b-9d5268b2a117 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:40.517479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed2734-8cec-4cf0-aed1-b2283edb1c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:40.518073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2734-8cec-4cf0-aed1-b2283edb1c75 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:40.518372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2734-8cec-4cf0-aed1-b2283edb1c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:40.518640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2734-8cec-4cf0-aed1-b2283edb1c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:40.528653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed2734-8cec-4cf0-aed1-b2283edb1c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:40.530074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed2734-8cec-4cf0-aed1-b2283edb1c75 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:40.539425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1127/2258] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:40.832454 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632be06f-ebcb-464d-89d0-74486abaeb46 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:40.834626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-632be06f-ebcb-464d-89d0-74486abaeb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:40.856211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632be06f-ebcb-464d-89d0-74486abaeb46 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:40.861679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:40.862485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:40.862625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:40.862885 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] PATH_INFO: `/v3/projects/d7cf1d0683bc47fba160946269925d48` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:40.864127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7cf1d0683bc47fba160946269925d48)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:40.866076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:40.926359 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7cf1d0683bc47fba160946269925d48', 'request_id': 'req-487403de-7dba-494d-a71b-23008c040fd8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.938527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7cf1d0683bc47fba160946269925d48', 'request_id': 'req-487403de-7dba-494d-a71b-23008c040fd8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.945395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7cf1d0683bc47fba160946269925d48', 'request_id': 'req-487403de-7dba-494d-a71b-23008c040fd8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.957222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] The token cache is being invalidate because project d7cf1d0683bc47fba160946269925d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:40.957591 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.958302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-487403de-7dba-494d-a71b-23008c040fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:40.960270 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1132/2259] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:40 2022] DELETE /identity/v3/projects/d7cf1d0683bc47fba160946269925d48 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:40.965552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e164ce-6421-44ef-a489-a6ad9c8b973d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:40.965930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e164ce-6421-44ef-a489-a6ad9c8b973d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:40.966299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e164ce-6421-44ef-a489-a6ad9c8b973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:40.988669 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-18e164ce-6421-44ef-a489-a6ad9c8b973d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:40.989931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-18e164ce-6421-44ef-a489-a6ad9c8b973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:41.020304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:41.066366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1128/2260] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:41.074968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be89ac-4e51-4de7-a6ec-ed343b7956fe None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.075253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be89ac-4e51-4de7-a6ec-ed343b7956fe None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.075522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be89ac-4e51-4de7-a6ec-ed343b7956fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.095084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-86be89ac-4e51-4de7-a6ec-ed343b7956fe None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:41.096637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-86be89ac-4e51-4de7-a6ec-ed343b7956fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:41.104383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1133/2261] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:41.108928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e164ce-6421-44ef-a489-a6ad9c8b973d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:41.119260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e164ce-6421-44ef-a489-a6ad9c8b973d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:41.124714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:41.125303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.125581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.125852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.126537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:41.126841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:41.128439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:41.144138 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1129/2262] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:41.148450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be89ac-4e51-4de7-a6ec-ed343b7956fe None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:41.157529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be89ac-4e51-4de7-a6ec-ed343b7956fe None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:41.162883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79801f87-6d27-4abf-8b2e-7f216fe64c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:41.163470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79801f87-6d27-4abf-8b2e-7f216fe64c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.163745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79801f87-6d27-4abf-8b2e-7f216fe64c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.164015 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79801f87-6d27-4abf-8b2e-7f216fe64c92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.164648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79801f87-6d27-4abf-8b2e-7f216fe64c92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:41.166778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79801f87-6d27-4abf-8b2e-7f216fe64c92 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:41.211085 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1134/2263] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:41 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:41.215467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:41.226369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ec8a4-bdf3-4a82-9053-dd143edf0c86 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:41.289251 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba7d2d-367e-4113-bda2-06a722d8d8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:41.289798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba7d2d-367e-4113-bda2-06a722d8d8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.290069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba7d2d-367e-4113-bda2-06a722d8d8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.290377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba7d2d-367e-4113-bda2-06a722d8d8c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.291280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba7d2d-367e-4113-bda2-06a722d8d8c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:41.293514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba7d2d-367e-4113-bda2-06a722d8d8c1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:41.315929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1130/2264] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:41 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:41.320527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79801f87-6d27-4abf-8b2e-7f216fe64c92 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:41.331450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79801f87-6d27-4abf-8b2e-7f216fe64c92 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:41.336839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819c84a-64f9-4640-ba73-d9ed974c5d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:41.337462 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c84a-64f9-4640-ba73-d9ed974c5d79 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.337824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c84a-64f9-4640-ba73-d9ed974c5d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.338160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c84a-64f9-4640-ba73-d9ed974c5d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.339052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819c84a-64f9-4640-ba73-d9ed974c5d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:41.340710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819c84a-64f9-4640-ba73-d9ed974c5d79 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:41.352050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1135/2265] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:41.357197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba7d2d-367e-4113-bda2-06a722d8d8c1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:41.367319 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba7d2d-367e-4113-bda2-06a722d8d8c1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:41.372643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1fde83-555a-4057-b050-0101118a14c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:41.373211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fde83-555a-4057-b050-0101118a14c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.373491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fde83-555a-4057-b050-0101118a14c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.373765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fde83-555a-4057-b050-0101118a14c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.374447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1fde83-555a-4057-b050-0101118a14c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:41.376060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1fde83-555a-4057-b050-0101118a14c9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:41.388657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1131/2266] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:41.394302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819c84a-64f9-4640-ba73-d9ed974c5d79 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:41.405141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819c84a-64f9-4640-ba73-d9ed974c5d79 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:41.410117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:41.410838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.411111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.411425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] PATH_INFO: `/v3/projects/b34afcf6799f4d84ada9a295d1aea7c8/users/2ce997cd838648ddaff729500e87f9b8/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.427171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34afcf6799f4d84ada9a295d1aea7c8, user_id=2ce997cd838648ddaff729500e87f9b8, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:41.429804 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:41.445192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1136/2267] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:41 2022] PUT /identity/v3/projects/b34afcf6799f4d84ada9a295d1aea7c8/users/2ce997cd838648ddaff729500e87f9b8/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:41.450978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f7ab-b31f-44a8-b8b4-b259ee5f3e5c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.451297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f7ab-b31f-44a8-b8b4-b259ee5f3e5c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.451562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f7ab-b31f-44a8-b8b4-b259ee5f3e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.471664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5ac5f7ab-b31f-44a8-b8b4-b259ee5f3e5c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:41.472898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ac5f7ab-b31f-44a8-b8b4-b259ee5f3e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:41.537512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1132/2268] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:41.547412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:41.549173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:41.554553 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:41.607921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35fd5a-16d8-4b79-819d-ecf31bd5ddf3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:41.611619 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:41.612223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:41.612500 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:41.612764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:41.623171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:41.624589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:41.633980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1137/2269] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:42.331713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56752-54a1-4dae-941c-8d5ee5ca9239 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:42.332227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56752-54a1-4dae-941c-8d5ee5ca9239 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:42.332227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56752-54a1-4dae-941c-8d5ee5ca9239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:42.363190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bec56752-54a1-4dae-941c-8d5ee5ca9239 None None] MFA Rules not processed for user `2ce997cd838648ddaff729500e87f9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:42.363992 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bec56752-54a1-4dae-941c-8d5ee5ca9239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:42.406366 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:42.444156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1133/2270] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:42.455157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:42.456605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:42.515576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c9981-dd7b-4c72-a7ef-2c27125ba179 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:42.519648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae3572-10da-47b7-ba25-d0fe23c8ab00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:42.520324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae3572-10da-47b7-ba25-d0fe23c8ab00 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:42.520640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae3572-10da-47b7-ba25-d0fe23c8ab00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:42.520913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae3572-10da-47b7-ba25-d0fe23c8ab00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:42.529509 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae3572-10da-47b7-ba25-d0fe23c8ab00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:42.530980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae3572-10da-47b7-ba25-d0fe23c8ab00 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:42.539620 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1138/2271] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:37:43.529217 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec56752-54a1-4dae-941c-8d5ee5ca9239 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:43.530722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bec56752-54a1-4dae-941c-8d5ee5ca9239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:43.544032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec56752-54a1-4dae-941c-8d5ee5ca9239 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:43.548794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:43.549398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:43.549667 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:43.549931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:43.551902 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:43.564578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:43.565990 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:43.575660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1134/2272] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:44.974863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569234-b84a-4f19-9607-322d1249b663 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:44.975718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569234-b84a-4f19-9607-322d1249b663 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:44.976136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569234-b84a-4f19-9607-322d1249b663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:44.997866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f9569234-b84a-4f19-9607-322d1249b663 None None] MFA Rules not processed for user `2ce997cd838648ddaff729500e87f9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:44.999471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f9569234-b84a-4f19-9607-322d1249b663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:45.008183 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1139/2273] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:45.053303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:45.054927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:45.127264 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801a6f3-0de8-4bb3-becf-f5bba1d94ae3 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:45.132525 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fd09151e284f639c93699b0d26db05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:45.133290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:45.133563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:45.133836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:45.136544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:45.256871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:45.258748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:45.287025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1135/2274] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:46.231316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9569234-b84a-4f19-9607-322d1249b663 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:46.244941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9569234-b84a-4f19-9607-322d1249b663 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:46.250065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:46.250970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:46.251339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:46.251832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:46.254851 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:46.273637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:46.275199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:46.289245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1140/2275] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:47.212025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:47.215147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:47.231166 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f2539-d12c-4210-869c-49334f7a3a00 service glance] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:47.235588 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:47.236219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:47.236492 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:47.236761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] PATH_INFO: `/v3/users/0e8717d368d44102bea0eb82fbee8a5c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:47.251361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8717d368d44102bea0eb82fbee8a5c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:47.253379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:47.274560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:47.304073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8717d368d44102bea0eb82fbee8a5c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.304760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8717d368d44102bea0eb82fbee8a5c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.316846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8717d368d44102bea0eb82fbee8a5c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.323726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8717d368d44102bea0eb82fbee8a5c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.324375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1296af19-2d5b-40e5-8560-298f9c509d8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8717d368d44102bea0eb82fbee8a5c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.338280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1136/2276] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:47 2022] DELETE /identity/v3/users/0e8717d368d44102bea0eb82fbee8a5c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:47.528349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:47.547224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638371bd-35f3-4ca1-b3e3-0023af58f90b service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:47.625119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:47.625893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:47.626224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:47.626494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] PATH_INFO: `/v3/projects/7402a850a25a48c48faf88fb4f3a56ae` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:47.638220 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7402a850a25a48c48faf88fb4f3a56ae)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:47.640630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:47.664661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:47.705869 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7402a850a25a48c48faf88fb4f3a56ae', 'request_id': 'req-5ffaf803-08dd-49b8-99cf-73029318e58d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.717761 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7402a850a25a48c48faf88fb4f3a56ae', 'request_id': 'req-5ffaf803-08dd-49b8-99cf-73029318e58d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.725955 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7402a850a25a48c48faf88fb4f3a56ae', 'request_id': 'req-5ffaf803-08dd-49b8-99cf-73029318e58d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.738516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] The token cache is being invalidate because project 7402a850a25a48c48faf88fb4f3a56ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:47.739042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.740077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ffaf803-08dd-49b8-99cf-73029318e58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:47.743075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1141/2277] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:47 2022] DELETE /identity/v3/projects/7402a850a25a48c48faf88fb4f3a56ae => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:47.748699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9d3c6-4203-42e2-88eb-4951c0c06f90 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:47.749190 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9d3c6-4203-42e2-88eb-4951c0c06f90 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:47.749640 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9d3c6-4203-42e2-88eb-4951c0c06f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:47.769822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7cb9d3c6-4203-42e2-88eb-4951c0c06f90 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:47.771071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cb9d3c6-4203-42e2-88eb-4951c0c06f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:47.839730 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1137/2278] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:47.848234 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8d6d5-9807-4e93-976d-0a48d67a72bc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:47.848504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8d6d5-9807-4e93-976d-0a48d67a72bc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:47.848807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8d6d5-9807-4e93-976d-0a48d67a72bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:47.871587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0ce8d6d5-9807-4e93-976d-0a48d67a72bc None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:47.872910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0ce8d6d5-9807-4e93-976d-0a48d67a72bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:47.885636 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1142/2279] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:47.886941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9d3c6-4203-42e2-88eb-4951c0c06f90 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:47.897612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9d3c6-4203-42e2-88eb-4951c0c06f90 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:47.904094 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:47.904745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:47.905029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:47.905302 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:47.905950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:47.906298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:47.907925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:47.921787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1138/2280] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:47.926609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8d6d5-9807-4e93-976d-0a48d67a72bc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:47.936865 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8d6d5-9807-4e93-976d-0a48d67a72bc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:47.941906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00450092-91ba-4090-a3d7-30a8baf0e3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:47.942896 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450092-91ba-4090-a3d7-30a8baf0e3ce admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:47.943142 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450092-91ba-4090-a3d7-30a8baf0e3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:47.943453 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450092-91ba-4090-a3d7-30a8baf0e3ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:47.944087 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450092-91ba-4090-a3d7-30a8baf0e3ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:47.946081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450092-91ba-4090-a3d7-30a8baf0e3ce admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:47.979678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1143/2281] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:47 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:47.981009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:47.991541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a04c5-6b2c-46a2-9c8d-e763cddc3ae5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.055092 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945fb7-60c3-4f1d-873f-d5a3b16f810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.055728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945fb7-60c3-4f1d-873f-d5a3b16f810a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.056000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945fb7-60c3-4f1d-873f-d5a3b16f810a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.056272 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945fb7-60c3-4f1d-873f-d5a3b16f810a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.056974 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd945fb7-60c3-4f1d-873f-d5a3b16f810a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.058919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd945fb7-60c3-4f1d-873f-d5a3b16f810a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.085998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1139/2282] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:47 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:48.088020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00450092-91ba-4090-a3d7-30a8baf0e3ce admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.099667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00450092-91ba-4090-a3d7-30a8baf0e3ce admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.104719 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993fa5e-8ab1-41fe-adcd-0406e79db5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.105340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993fa5e-8ab1-41fe-adcd-0406e79db5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.105618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993fa5e-8ab1-41fe-adcd-0406e79db5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.105911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993fa5e-8ab1-41fe-adcd-0406e79db5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.106589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993fa5e-8ab1-41fe-adcd-0406e79db5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.108212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993fa5e-8ab1-41fe-adcd-0406e79db5e0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.118877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1144/2283] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:48.123090 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945fb7-60c3-4f1d-873f-d5a3b16f810a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.133359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945fb7-60c3-4f1d-873f-d5a3b16f810a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.138514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314b976-33ce-425b-8bcb-22a0088b9939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.139062 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314b976-33ce-425b-8bcb-22a0088b9939 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.139332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314b976-33ce-425b-8bcb-22a0088b9939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.139600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314b976-33ce-425b-8bcb-22a0088b9939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.140274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314b976-33ce-425b-8bcb-22a0088b9939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.141963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314b976-33ce-425b-8bcb-22a0088b9939 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.153180 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1140/2284] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:48.157429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993fa5e-8ab1-41fe-adcd-0406e79db5e0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.167382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993fa5e-8ab1-41fe-adcd-0406e79db5e0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.172240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a9b91-08d5-467d-91a3-7fe76e97a137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.172979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a9b91-08d5-467d-91a3-7fe76e97a137 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.173255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a9b91-08d5-467d-91a3-7fe76e97a137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.173525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a9b91-08d5-467d-91a3-7fe76e97a137 admin admin] PATH_INFO: `/v3/projects/17099f38a2064dae9cf29e2902fbdf46/users/a38f802f62c44d939cc968489f77045b/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.189797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a9b91-08d5-467d-91a3-7fe76e97a137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17099f38a2064dae9cf29e2902fbdf46, user_id=a38f802f62c44d939cc968489f77045b, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.192471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a9b91-08d5-467d-91a3-7fe76e97a137 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.207214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1145/2285] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:48 2022] PUT /identity/v3/projects/17099f38a2064dae9cf29e2902fbdf46/users/a38f802f62c44d939cc968489f77045b/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:48.218740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a46c1a-69d8-47e1-a07e-664320ae427d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.219287 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a46c1a-69d8-47e1-a07e-664320ae427d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.219396 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a46c1a-69d8-47e1-a07e-664320ae427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.232321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:48.258346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f0a46c1a-69d8-47e1-a07e-664320ae427d None None] MFA Rules not processed for user `a38f802f62c44d939cc968489f77045b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:48.259709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f0a46c1a-69d8-47e1-a07e-664320ae427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:48.333987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1141/2286] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:48.338516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a9b91-08d5-467d-91a3-7fe76e97a137 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.417838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a9b91-08d5-467d-91a3-7fe76e97a137 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.428425 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304685cd-0a22-47f7-8866-16556b1c250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.429263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304685cd-0a22-47f7-8866-16556b1c250e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.429690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304685cd-0a22-47f7-8866-16556b1c250e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.429997 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304685cd-0a22-47f7-8866-16556b1c250e admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.431237 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304685cd-0a22-47f7-8866-16556b1c250e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.433766 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304685cd-0a22-47f7-8866-16556b1c250e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.489967 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1146/2287] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:48 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:48.494847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a46c1a-69d8-47e1-a07e-664320ae427d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.510340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a46c1a-69d8-47e1-a07e-664320ae427d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.582869 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8458d-0bf5-4d7e-a148-ffb567552769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.583572 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8458d-0bf5-4d7e-a148-ffb567552769 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.583908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8458d-0bf5-4d7e-a148-ffb567552769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.584239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8458d-0bf5-4d7e-a148-ffb567552769 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.584938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8458d-0bf5-4d7e-a148-ffb567552769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.587111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8458d-0bf5-4d7e-a148-ffb567552769 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.609782 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1142/2288] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:48 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:48.614039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304685cd-0a22-47f7-8866-16556b1c250e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.625526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304685cd-0a22-47f7-8866-16556b1c250e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.631988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79d3cb-f1ee-4352-ae8a-cfdc5fd8ee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.632602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79d3cb-f1ee-4352-ae8a-cfdc5fd8ee58 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.632909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79d3cb-f1ee-4352-ae8a-cfdc5fd8ee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.633179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79d3cb-f1ee-4352-ae8a-cfdc5fd8ee58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.633852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79d3cb-f1ee-4352-ae8a-cfdc5fd8ee58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.635554 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79d3cb-f1ee-4352-ae8a-cfdc5fd8ee58 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.653880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1147/2289] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:48.655305 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8458d-0bf5-4d7e-a148-ffb567552769 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.667165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8458d-0bf5-4d7e-a148-ffb567552769 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.671985 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1badb67-9788-4a14-a9d2-c5748d6125a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.672611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1badb67-9788-4a14-a9d2-c5748d6125a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.673039 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1badb67-9788-4a14-a9d2-c5748d6125a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.673311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1badb67-9788-4a14-a9d2-c5748d6125a7 admin admin] PATH_INFO: `/v3/projects/6bb2077e0a5649b79508603a297485fd/users/d32de18b4bff49cbb689394f3243c354/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.691398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1badb67-9788-4a14-a9d2-c5748d6125a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb2077e0a5649b79508603a297485fd, user_id=d32de18b4bff49cbb689394f3243c354, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.694244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1badb67-9788-4a14-a9d2-c5748d6125a7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.710982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1143/2290] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:48 2022] PUT /identity/v3/projects/6bb2077e0a5649b79508603a297485fd/users/d32de18b4bff49cbb689394f3243c354/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:48.715126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79d3cb-f1ee-4352-ae8a-cfdc5fd8ee58 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.725404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79d3cb-f1ee-4352-ae8a-cfdc5fd8ee58 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.744342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:48.792442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63489c7a-f3b5-48cb-a577-eba903373b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.793171 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63489c7a-f3b5-48cb-a577-eba903373b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.793515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63489c7a-f3b5-48cb-a577-eba903373b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.793849 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63489c7a-f3b5-48cb-a577-eba903373b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.794616 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63489c7a-f3b5-48cb-a577-eba903373b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.796372 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63489c7a-f3b5-48cb-a577-eba903373b71 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.807711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1148/2291] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:48.811819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1badb67-9788-4a14-a9d2-c5748d6125a7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.822443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1badb67-9788-4a14-a9d2-c5748d6125a7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:48.833384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:48.834061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.834377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.834652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] PATH_INFO: `/v3/projects/6bb2077e0a5649b79508603a297485fd/users/d32de18b4bff49cbb689394f3243c354/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.836584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb2077e0a5649b79508603a297485fd, user_id=d32de18b4bff49cbb689394f3243c354, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:48.839568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:48.850987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1144/2292] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:48 2022] PUT /identity/v3/projects/6bb2077e0a5649b79508603a297485fd/users/d32de18b4bff49cbb689394f3243c354/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:48.862019 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf465a-3d7a-43ce-b2d4-34127a911688 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:48.862398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf465a-3d7a-43ce-b2d4-34127a911688 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:48.862737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf465a-3d7a-43ce-b2d4-34127a911688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:48.891116 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4dcf465a-3d7a-43ce-b2d4-34127a911688 None None] MFA Rules not processed for user `d32de18b4bff49cbb689394f3243c354`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:48.892325 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4dcf465a-3d7a-43ce-b2d4-34127a911688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:48.967444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1149/2293] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:48.979641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:48.981346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:49.030847 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:49.044184 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517211c9-97c7-4078-8ac1-0138ce454303 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:49.047989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb67c7-0aa3-47f3-b241-5147ed33057f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:49.048624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb67c7-0aa3-47f3-b241-5147ed33057f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:49.049283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb67c7-0aa3-47f3-b241-5147ed33057f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:49.049704 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb67c7-0aa3-47f3-b241-5147ed33057f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:49.059729 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb67c7-0aa3-47f3-b241-5147ed33057f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:49.061239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb67c7-0aa3-47f3-b241-5147ed33057f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:49.070850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1145/2294] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:49.482663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf465a-3d7a-43ce-b2d4-34127a911688 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:49.561296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf465a-3d7a-43ce-b2d4-34127a911688 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:49.568395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:49.569038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:49.569891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:49.570216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] PATH_INFO: `/v3/users/a38f802f62c44d939cc968489f77045b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:49.571635 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a38f802f62c44d939cc968489f77045b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:49.574003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:49.633301 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:49.640503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a38f802f62c44d939cc968489f77045b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:49.641402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a38f802f62c44d939cc968489f77045b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:49.655629 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a38f802f62c44d939cc968489f77045b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:49.662520 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a38f802f62c44d939cc968489f77045b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:49.663303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a38f802f62c44d939cc968489f77045b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:49.676909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1150/2295] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:49 2022] DELETE /identity/v3/users/a38f802f62c44d939cc968489f77045b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:49.681491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74625-1b9d-43ae-a465-12013626c6ee None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:49.682024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74625-1b9d-43ae-a465-12013626c6ee None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:49.682024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74625-1b9d-43ae-a465-12013626c6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:49.704116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d4e74625-1b9d-43ae-a465-12013626c6ee None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:49.705336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d4e74625-1b9d-43ae-a465-12013626c6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:49.773659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1146/2296] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:49.780397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:49.781939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:49.850922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82927-ff5c-4398-895f-5273fcae9b95 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:49.854889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70375e61-2ddc-431e-abf1-f832eaefdaf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:49.855642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70375e61-2ddc-431e-abf1-f832eaefdaf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:49.855946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70375e61-2ddc-431e-abf1-f832eaefdaf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:49.856179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70375e61-2ddc-431e-abf1-f832eaefdaf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:49.866345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70375e61-2ddc-431e-abf1-f832eaefdaf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:49.867812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70375e61-2ddc-431e-abf1-f832eaefdaf1 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:49.878156 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1151/2297] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:50.164277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74625-1b9d-43ae-a465-12013626c6ee None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:50.165870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d4e74625-1b9d-43ae-a465-12013626c6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:50.179759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74625-1b9d-43ae-a465-12013626c6ee None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:50.185392 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:50.185966 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:50.186277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:50.186562 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] PATH_INFO: `/v3/projects/17099f38a2064dae9cf29e2902fbdf46` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:50.187751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17099f38a2064dae9cf29e2902fbdf46)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:50.189657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:50.238308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17099f38a2064dae9cf29e2902fbdf46', 'request_id': 'req-ec787dc4-06d1-4978-b51a-8fac54156605'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.244668 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17099f38a2064dae9cf29e2902fbdf46', 'request_id': 'req-ec787dc4-06d1-4978-b51a-8fac54156605'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.252925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17099f38a2064dae9cf29e2902fbdf46', 'request_id': 'req-ec787dc4-06d1-4978-b51a-8fac54156605'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.263367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] The token cache is being invalidate because project 17099f38a2064dae9cf29e2902fbdf46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:50.263650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.264259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.265722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1147/2298] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:50 2022] DELETE /identity/v3/projects/17099f38a2064dae9cf29e2902fbdf46 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:50.269826 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70375e61-2ddc-431e-abf1-f832eaefdaf1 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:50.279445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70375e61-2ddc-431e-abf1-f832eaefdaf1 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:50.337348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:50.338113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:50.338383 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:50.338649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] PATH_INFO: `/v3/users/d32de18b4bff49cbb689394f3243c354` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:50.339787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32de18b4bff49cbb689394f3243c354)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:50.341799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:50.392094 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32de18b4bff49cbb689394f3243c354'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.392785 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32de18b4bff49cbb689394f3243c354'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.404166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32de18b4bff49cbb689394f3243c354'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.405493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:50.413337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32de18b4bff49cbb689394f3243c354'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.414312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eb2ac120-79fb-44e5-9983-ca0c5b9726d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32de18b4bff49cbb689394f3243c354'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.428481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1152/2299] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:50 2022] DELETE /identity/v3/users/d32de18b4bff49cbb689394f3243c354 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:50.726642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:50.732086 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:50.757124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec787dc4-06d1-4978-b51a-8fac54156605 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:50.820627 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:50.821253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:50.821526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:50.821791 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] PATH_INFO: `/v3/projects/6bb2077e0a5649b79508603a297485fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:50.823093 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb2077e0a5649b79508603a297485fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:50.824987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:50.836145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:50.873828 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb2077e0a5649b79508603a297485fd', 'request_id': 'req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.879815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb2077e0a5649b79508603a297485fd', 'request_id': 'req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.887652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb2077e0a5649b79508603a297485fd', 'request_id': 'req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.903229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] The token cache is being invalidate because project 6bb2077e0a5649b79508603a297485fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:50.903505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.904296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a03a6e2c-f0b7-4a21-a304-3a68ff68bc8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:50.905999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1148/2300] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:50 2022] DELETE /identity/v3/projects/6bb2077e0a5649b79508603a297485fd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:50.911903 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716813-e54a-4bee-bf24-9b6b090838e7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:50.912421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716813-e54a-4bee-bf24-9b6b090838e7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:50.912421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716813-e54a-4bee-bf24-9b6b090838e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:50.932931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-54716813-e54a-4bee-bf24-9b6b090838e7 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:50.934516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-54716813-e54a-4bee-bf24-9b6b090838e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:50.997233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1153/2301] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:51.005290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8debf4f-14c9-4c7a-a93b-e42171fe6cc2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.005578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8debf4f-14c9-4c7a-a93b-e42171fe6cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.005851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8debf4f-14c9-4c7a-a93b-e42171fe6cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.027809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d8debf4f-14c9-4c7a-a93b-e42171fe6cc2 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:51.029031 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8debf4f-14c9-4c7a-a93b-e42171fe6cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:51.038321 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1149/2302] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:51.042756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54716813-e54a-4bee-bf24-9b6b090838e7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.053434 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54716813-e54a-4bee-bf24-9b6b090838e7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.064887 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.065547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.065798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.066060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.066860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:51.067304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.068945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.083467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1154/2303] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:51.088145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8debf4f-14c9-4c7a-a93b-e42171fe6cc2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.098259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8debf4f-14c9-4c7a-a93b-e42171fe6cc2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.103474 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fffc2-fdf2-40ed-b0d4-86fd1601ec57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.104216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fffc2-fdf2-40ed-b0d4-86fd1601ec57 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.104485 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fffc2-fdf2-40ed-b0d4-86fd1601ec57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.104754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fffc2-fdf2-40ed-b0d4-86fd1601ec57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.105431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fffc2-fdf2-40ed-b0d4-86fd1601ec57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.107663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fffc2-fdf2-40ed-b0d4-86fd1601ec57 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.139040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1150/2304] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:51 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:51.140231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.150364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf375f76-a9e8-45f2-ad97-05a0ba179c14 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.211904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcca7d6-22e6-41b7-bf2d-4258815cc834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.212524 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca7d6-22e6-41b7-bf2d-4258815cc834 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.212799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca7d6-22e6-41b7-bf2d-4258815cc834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.213105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca7d6-22e6-41b7-bf2d-4258815cc834 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.213737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcca7d6-22e6-41b7-bf2d-4258815cc834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.215693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcca7d6-22e6-41b7-bf2d-4258815cc834 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.238563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1155/2305] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:51 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:37:51.242571 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fffc2-fdf2-40ed-b0d4-86fd1601ec57 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.253206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fffc2-fdf2-40ed-b0d4-86fd1601ec57 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.258854 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b18669-51ae-4fbe-b5c2-561ba1efec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.259489 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b18669-51ae-4fbe-b5c2-561ba1efec74 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.259765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b18669-51ae-4fbe-b5c2-561ba1efec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.260112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b18669-51ae-4fbe-b5c2-561ba1efec74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.260768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b18669-51ae-4fbe-b5c2-561ba1efec74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.262429 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b18669-51ae-4fbe-b5c2-561ba1efec74 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.272661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1151/2306] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:51.277260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcca7d6-22e6-41b7-bf2d-4258815cc834 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.288837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcca7d6-22e6-41b7-bf2d-4258815cc834 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.294441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda68f02-ecdd-4669-80c6-c2058cfc4c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.295038 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda68f02-ecdd-4669-80c6-c2058cfc4c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.295319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda68f02-ecdd-4669-80c6-c2058cfc4c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.295590 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda68f02-ecdd-4669-80c6-c2058cfc4c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.296231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda68f02-ecdd-4669-80c6-c2058cfc4c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.297837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda68f02-ecdd-4669-80c6-c2058cfc4c2b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.308853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1156/2307] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:51.313295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b18669-51ae-4fbe-b5c2-561ba1efec74 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.323639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b18669-51ae-4fbe-b5c2-561ba1efec74 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.328753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd25d-777a-4dea-8e06-88e4ba2223cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.329404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd25d-777a-4dea-8e06-88e4ba2223cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.329687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd25d-777a-4dea-8e06-88e4ba2223cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.329961 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd25d-777a-4dea-8e06-88e4ba2223cc admin admin] PATH_INFO: `/v3/projects/5b49c3f3e37c48d38c5a7249a6a82422/users/38b153b2f4b84a3d8c5a90a46b63af32/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.345342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cd25d-777a-4dea-8e06-88e4ba2223cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b49c3f3e37c48d38c5a7249a6a82422, user_id=38b153b2f4b84a3d8c5a90a46b63af32, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.348378 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cd25d-777a-4dea-8e06-88e4ba2223cc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.363229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1152/2308] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:51 2022] PUT /identity/v3/projects/5b49c3f3e37c48d38c5a7249a6a82422/users/38b153b2f4b84a3d8c5a90a46b63af32/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:51.375066 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff255076-5653-48c5-bfbc-819a98f49f19 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.375356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff255076-5653-48c5-bfbc-819a98f49f19 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.375633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff255076-5653-48c5-bfbc-819a98f49f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.404917 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ff255076-5653-48c5-bfbc-819a98f49f19 None None] MFA Rules not processed for user `38b153b2f4b84a3d8c5a90a46b63af32`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:51.406195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ff255076-5653-48c5-bfbc-819a98f49f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:51.449108 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:51.482573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1157/2309] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:51.487262 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd25d-777a-4dea-8e06-88e4ba2223cc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.551331 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd25d-777a-4dea-8e06-88e4ba2223cc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.557819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917e8af-18b6-4442-93a8-b2a135e760f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.558397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e8af-18b6-4442-93a8-b2a135e760f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.558643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e8af-18b6-4442-93a8-b2a135e760f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.558905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e8af-18b6-4442-93a8-b2a135e760f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.559549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917e8af-18b6-4442-93a8-b2a135e760f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.561453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917e8af-18b6-4442-93a8-b2a135e760f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.592170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1153/2310] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:51 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:51.596687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff255076-5653-48c5-bfbc-819a98f49f19 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.608211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff255076-5653-48c5-bfbc-819a98f49f19 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.665391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86718f8-7484-48ab-afed-612527d0a88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.665986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86718f8-7484-48ab-afed-612527d0a88d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.666521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86718f8-7484-48ab-afed-612527d0a88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.666521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86718f8-7484-48ab-afed-612527d0a88d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.667415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86718f8-7484-48ab-afed-612527d0a88d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.669610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86718f8-7484-48ab-afed-612527d0a88d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.698614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1158/2311] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:51 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:51.703446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917e8af-18b6-4442-93a8-b2a135e760f8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.715876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917e8af-18b6-4442-93a8-b2a135e760f8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.722068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81931ba-7ef4-4911-a0b3-a442ee5f2a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.722688 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81931ba-7ef4-4911-a0b3-a442ee5f2a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.722962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81931ba-7ef4-4911-a0b3-a442ee5f2a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.723223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81931ba-7ef4-4911-a0b3-a442ee5f2a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.723868 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81931ba-7ef4-4911-a0b3-a442ee5f2a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.725467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81931ba-7ef4-4911-a0b3-a442ee5f2a4c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.736159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1154/2312] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:51.740401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86718f8-7484-48ab-afed-612527d0a88d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.754724 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86718f8-7484-48ab-afed-612527d0a88d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.760006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.760671 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.760929 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.761217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] PATH_INFO: `/v3/projects/b5812aeb630f4409b08cef0e3e24e061/users/26e0ca01c2b04a02b28398b4c2620939/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.776812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5812aeb630f4409b08cef0e3e24e061, user_id=26e0ca01c2b04a02b28398b4c2620939, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.779486 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.796531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1159/2313] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:51 2022] PUT /identity/v3/projects/b5812aeb630f4409b08cef0e3e24e061/users/26e0ca01c2b04a02b28398b4c2620939/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:51.803224 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81931ba-7ef4-4911-a0b3-a442ee5f2a4c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.814623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81931ba-7ef4-4911-a0b3-a442ee5f2a4c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.819570 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.821340 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:51.824244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:51.875591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7b73b-c837-443f-94f6-4229650ee690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.876192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7b73b-c837-443f-94f6-4229650ee690 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.876466 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7b73b-c837-443f-94f6-4229650ee690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.876733 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7b73b-c837-443f-94f6-4229650ee690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.877377 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7b73b-c837-443f-94f6-4229650ee690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.878995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7b73b-c837-443f-94f6-4229650ee690 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.889223 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1155/2314] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:51.889593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe04919-9adf-4596-b460-42ef2f8e4f5c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.893356 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7b73b-c837-443f-94f6-4229650ee690 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:51.894452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.895091 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.895371 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.895644 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] PATH_INFO: `/v3/users/e4627110f092433694313fea1847475f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.896836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4627110f092433694313fea1847475f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.898814 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.903701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7b73b-c837-443f-94f6-4229650ee690 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:51.909016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954c19a-e257-47f0-8633-81cb68003b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:51.909664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954c19a-e257-47f0-8633-81cb68003b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.909949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954c19a-e257-47f0-8633-81cb68003b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.910244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954c19a-e257-47f0-8633-81cb68003b9a admin admin] PATH_INFO: `/v3/projects/b5812aeb630f4409b08cef0e3e24e061/users/26e0ca01c2b04a02b28398b4c2620939/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.912383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954c19a-e257-47f0-8633-81cb68003b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5812aeb630f4409b08cef0e3e24e061, user_id=26e0ca01c2b04a02b28398b4c2620939, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:51.915051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954c19a-e257-47f0-8633-81cb68003b9a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:51.926408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1156/2315] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:51 2022] PUT /identity/v3/projects/b5812aeb630f4409b08cef0e3e24e061/users/26e0ca01c2b04a02b28398b4c2620939/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:51.937641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f0cef-0c8a-433c-8784-49fe21eee16e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:51.938374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f0cef-0c8a-433c-8784-49fe21eee16e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:51.938650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f0cef-0c8a-433c-8784-49fe21eee16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:51.955654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:51.957474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4627110f092433694313fea1847475f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:51.958150 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4627110f092433694313fea1847475f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:51.967481 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e59f0cef-0c8a-433c-8784-49fe21eee16e None None] MFA Rules not processed for user `26e0ca01c2b04a02b28398b4c2620939`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:51.968695 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e59f0cef-0c8a-433c-8784-49fe21eee16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:51.972221 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4627110f092433694313fea1847475f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:51.979070 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4627110f092433694313fea1847475f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:51.979756 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-859ba176-d2fb-4540-8ff6-6e1667310614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4627110f092433694313fea1847475f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:51.995718 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1160/2316] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:51 2022] DELETE /identity/v3/users/e4627110f092433694313fea1847475f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:52.000061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d322-65ab-4763-9058-c790071f46f3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:52.000489 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d322-65ab-4763-9058-c790071f46f3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:52.000608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d322-65ab-4763-9058-c790071f46f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:52.019356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1ee4d322-65ab-4763-9058-c790071f46f3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:52.020548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1ee4d322-65ab-4763-9058-c790071f46f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:52.041338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1157/2317] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:52.052197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f0cef-0c8a-433c-8784-49fe21eee16e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:52.053537 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e59f0cef-0c8a-433c-8784-49fe21eee16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:52.079943 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1161/2318] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:52.090746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d322-65ab-4763-9058-c790071f46f3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:52.092235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1ee4d322-65ab-4763-9058-c790071f46f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:52.111803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f0cef-0c8a-433c-8784-49fe21eee16e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:52.144483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:52.148909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d322-65ab-4763-9058-c790071f46f3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:52.153308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:52.153984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:52.154690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:52.154884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:52.161706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:52.162334 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:52.162608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:52.162878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:52.164128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:52.165622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:52.175176 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1162/2319] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:52.224178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:52.226033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:52.237155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1158/2320] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:37:52.731393 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:52.732828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:52.755638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823b7d-c35d-4d7c-bab5-1902bc7844f1 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:52.760891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:52.761709 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:52.762465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:52.762719 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] PATH_INFO: `/v3/projects/ab8f66bca550493798a7aee3a189fb8b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:52.764020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab8f66bca550493798a7aee3a189fb8b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:52.766064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:52.822736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab8f66bca550493798a7aee3a189fb8b', 'request_id': 'req-cf4acb51-3414-446c-be38-95629d9d24b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:52.833587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab8f66bca550493798a7aee3a189fb8b', 'request_id': 'req-cf4acb51-3414-446c-be38-95629d9d24b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:52.841441 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab8f66bca550493798a7aee3a189fb8b', 'request_id': 'req-cf4acb51-3414-446c-be38-95629d9d24b9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:52.843060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:52.856908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] The token cache is being invalidate because project ab8f66bca550493798a7aee3a189fb8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:52.857202 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:52.857943 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-cf4acb51-3414-446c-be38-95629d9d24b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:52.859991 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1163/2321] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:52 2022] DELETE /identity/v3/projects/ab8f66bca550493798a7aee3a189fb8b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:53.725972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:53.727448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:53.742228 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd15b-e88e-458f-809d-852087c716c2 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:53.809472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:53.810109 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:53.810406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:53.810674 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] PATH_INFO: `/v3/users/38b153b2f4b84a3d8c5a90a46b63af32` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:53.811883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b153b2f4b84a3d8c5a90a46b63af32)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:53.813839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:53.866227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b153b2f4b84a3d8c5a90a46b63af32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:53.866885 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b153b2f4b84a3d8c5a90a46b63af32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:53.879661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b153b2f4b84a3d8c5a90a46b63af32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:53.887258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b153b2f4b84a3d8c5a90a46b63af32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:53.887900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b153b2f4b84a3d8c5a90a46b63af32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:53.904308 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1159/2322] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:53 2022] DELETE /identity/v3/users/38b153b2f4b84a3d8c5a90a46b63af32 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:53.908687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac23a2-2687-47b1-a10a-698ce3886829 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:53.909203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac23a2-2687-47b1-a10a-698ce3886829 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:53.909203 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac23a2-2687-47b1-a10a-698ce3886829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:53.929158 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-afac23a2-2687-47b1-a10a-698ce3886829 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:53.930389 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-afac23a2-2687-47b1-a10a-698ce3886829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:53.990828 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1164/2323] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:54.001011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:54.002461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:54.009053 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:54.065093 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9f85-eb42-4ca5-b798-492821523b62 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:54.068865 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75f1da-906a-4d8d-965b-eda8070e8f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:54.069475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f1da-906a-4d8d-965b-eda8070e8f04 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:54.069751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f1da-906a-4d8d-965b-eda8070e8f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:54.070020 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f1da-906a-4d8d-965b-eda8070e8f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:54.079563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75f1da-906a-4d8d-965b-eda8070e8f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:54.081025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75f1da-906a-4d8d-965b-eda8070e8f04 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:54.090950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1160/2324] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:54.375375 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac23a2-2687-47b1-a10a-698ce3886829 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:54.377052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-afac23a2-2687-47b1-a10a-698ce3886829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:54.391550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac23a2-2687-47b1-a10a-698ce3886829 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:54.396789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:54.397460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:54.397700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:54.397969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] PATH_INFO: `/v3/projects/5b49c3f3e37c48d38c5a7249a6a82422` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:54.399136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b49c3f3e37c48d38c5a7249a6a82422)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:54.401036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:54.448532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b49c3f3e37c48d38c5a7249a6a82422', 'request_id': 'req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.457353 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b49c3f3e37c48d38c5a7249a6a82422', 'request_id': 'req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.463807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b49c3f3e37c48d38c5a7249a6a82422', 'request_id': 'req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.474503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] The token cache is being invalidate because project 5b49c3f3e37c48d38c5a7249a6a82422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:54.474783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.475411 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.476889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1165/2325] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:54 2022] DELETE /identity/v3/projects/5b49c3f3e37c48d38c5a7249a6a82422 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:54.481322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75f1da-906a-4d8d-965b-eda8070e8f04 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:54.491362 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75f1da-906a-4d8d-965b-eda8070e8f04 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:54.552752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:54.553394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:54.553682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:54.553949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] PATH_INFO: `/v3/users/26e0ca01c2b04a02b28398b4c2620939` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:54.555112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e0ca01c2b04a02b28398b4c2620939)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:54.557037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:54.610110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e0ca01c2b04a02b28398b4c2620939'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.610774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e0ca01c2b04a02b28398b4c2620939'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.622711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e0ca01c2b04a02b28398b4c2620939'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.629421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e0ca01c2b04a02b28398b4c2620939'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.630058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3ff95285-a966-416d-89cb-50af4500e778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e0ca01c2b04a02b28398b4c2620939'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:54.644441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1161/2326] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:54 2022] DELETE /identity/v3/users/26e0ca01c2b04a02b28398b4c2620939 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:54.933046 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:54.938396 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:54.948364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2c2dd-7a28-4b73-a012-1f00304d5e75 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.006713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.007268 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.007538 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.007809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] PATH_INFO: `/v3/projects/b5812aeb630f4409b08cef0e3e24e061` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.008950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5812aeb630f4409b08cef0e3e24e061)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.010902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.026924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:55.060122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5812aeb630f4409b08cef0e3e24e061', 'request_id': 'req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:55.069151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5812aeb630f4409b08cef0e3e24e061', 'request_id': 'req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:55.075739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5812aeb630f4409b08cef0e3e24e061', 'request_id': 'req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:55.086367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] The token cache is being invalidate because project b5812aeb630f4409b08cef0e3e24e061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:55.086646 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:55.087310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-83ef226f-41dd-49c0-a0e3-a5c11670ceaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:55.088790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1166/2327] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:54 2022] DELETE /identity/v3/projects/b5812aeb630f4409b08cef0e3e24e061 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:55.094112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb4ee8-66bc-44d3-b51c-4ead832104ef None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.094642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb4ee8-66bc-44d3-b51c-4ead832104ef None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.094642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb4ee8-66bc-44d3-b51c-4ead832104ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.115965 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-60cb4ee8-66bc-44d3-b51c-4ead832104ef None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:55.117163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-60cb4ee8-66bc-44d3-b51c-4ead832104ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:55.138292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:55.185482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1162/2328] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:37:55.193404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d70bc-39e0-4a95-be39-2bdd58cba0c1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.193669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d70bc-39e0-4a95-be39-2bdd58cba0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.193935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d70bc-39e0-4a95-be39-2bdd58cba0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.213974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8a2d70bc-39e0-4a95-be39-2bdd58cba0c1 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:55.215339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8a2d70bc-39e0-4a95-be39-2bdd58cba0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:55.224319 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1167/2329] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:55.228771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb4ee8-66bc-44d3-b51c-4ead832104ef None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.239353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb4ee8-66bc-44d3-b51c-4ead832104ef None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.244586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.245181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.245451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.245709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.246400 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:55.246701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.248294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.259898 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1163/2330] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:37:55.264752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d70bc-39e0-4a95-be39-2bdd58cba0c1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.278307 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d70bc-39e0-4a95-be39-2bdd58cba0c1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.284532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473761dc-1cec-4577-bfde-061782e70331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.285124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473761dc-1cec-4577-bfde-061782e70331 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.285395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473761dc-1cec-4577-bfde-061782e70331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.285663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473761dc-1cec-4577-bfde-061782e70331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.286300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473761dc-1cec-4577-bfde-061782e70331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.288205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473761dc-1cec-4577-bfde-061782e70331 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.319227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1168/2331] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:55 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:55.323598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.335340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d1c40-c3ac-4082-8428-5a508a8f89b8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.408495 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfd5d3-0c9f-4f10-9776-97e381b2fc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.409108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfd5d3-0c9f-4f10-9776-97e381b2fc84 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.409383 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfd5d3-0c9f-4f10-9776-97e381b2fc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.409649 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfd5d3-0c9f-4f10-9776-97e381b2fc84 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.410330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfd5d3-0c9f-4f10-9776-97e381b2fc84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.412255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfd5d3-0c9f-4f10-9776-97e381b2fc84 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.437059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1164/2332] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:55 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:55.441610 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473761dc-1cec-4577-bfde-061782e70331 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.453263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473761dc-1cec-4577-bfde-061782e70331 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.458483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072ff8d-897c-407a-a07a-34b16eab0556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.459095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072ff8d-897c-407a-a07a-34b16eab0556 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.459373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072ff8d-897c-407a-a07a-34b16eab0556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.459642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072ff8d-897c-407a-a07a-34b16eab0556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.460289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072ff8d-897c-407a-a07a-34b16eab0556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.461905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072ff8d-897c-407a-a07a-34b16eab0556 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.473549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1169/2333] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:55.477856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfd5d3-0c9f-4f10-9776-97e381b2fc84 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.488655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfd5d3-0c9f-4f10-9776-97e381b2fc84 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.493936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b06cc-425c-4fcb-9e78-417422057dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.494576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b06cc-425c-4fcb-9e78-417422057dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.494856 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b06cc-425c-4fcb-9e78-417422057dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.495125 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b06cc-425c-4fcb-9e78-417422057dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.495893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b06cc-425c-4fcb-9e78-417422057dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.497514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b06cc-425c-4fcb-9e78-417422057dbd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.509163 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1165/2334] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:37:55.514184 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072ff8d-897c-407a-a07a-34b16eab0556 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.525642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072ff8d-897c-407a-a07a-34b16eab0556 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.531777 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7c3e-af4a-4632-b39b-99b9ebfa8e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.532567 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7c3e-af4a-4632-b39b-99b9ebfa8e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.532795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7c3e-af4a-4632-b39b-99b9ebfa8e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.533039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7c3e-af4a-4632-b39b-99b9ebfa8e89 admin admin] PATH_INFO: `/v3/projects/1fc6fdf5e9484a0c9076cb5bb7ea368e/users/72c569f3d5864d7e93c66ea4a39f295e/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.553661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a7c3e-af4a-4632-b39b-99b9ebfa8e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc6fdf5e9484a0c9076cb5bb7ea368e, user_id=72c569f3d5864d7e93c66ea4a39f295e, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.557211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a7c3e-af4a-4632-b39b-99b9ebfa8e89 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.570768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1170/2335] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:55 2022] PUT /identity/v3/projects/1fc6fdf5e9484a0c9076cb5bb7ea368e/users/72c569f3d5864d7e93c66ea4a39f295e/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:55.582437 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e47c-32bb-4740-9051-5a77e43297ca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.582719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e47c-32bb-4740-9051-5a77e43297ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.582988 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e47c-32bb-4740-9051-5a77e43297ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.613606 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3a81e47c-32bb-4740-9051-5a77e43297ca None None] MFA Rules not processed for user `72c569f3d5864d7e93c66ea4a39f295e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:55.614871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3a81e47c-32bb-4740-9051-5a77e43297ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:55.685431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1166/2336] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:55.690460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7c3e-af4a-4632-b39b-99b9ebfa8e89 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.755745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7c3e-af4a-4632-b39b-99b9ebfa8e89 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.760449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d203cc-4cc0-4c5d-bd86-70ac9fb31128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.761058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d203cc-4cc0-4c5d-bd86-70ac9fb31128 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.761435 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d203cc-4cc0-4c5d-bd86-70ac9fb31128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.762328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d203cc-4cc0-4c5d-bd86-70ac9fb31128 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.763197 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d203cc-4cc0-4c5d-bd86-70ac9fb31128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.765279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d203cc-4cc0-4c5d-bd86-70ac9fb31128 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.793470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1171/2337] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:55 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:55.797543 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81e47c-32bb-4740-9051-5a77e43297ca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.809960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81e47c-32bb-4740-9051-5a77e43297ca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.830271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:55.871353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947b2e2-5494-416a-8a8d-47ff53f535bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.871963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947b2e2-5494-416a-8a8d-47ff53f535bc admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.872241 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947b2e2-5494-416a-8a8d-47ff53f535bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.872511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947b2e2-5494-416a-8a8d-47ff53f535bc admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.873137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947b2e2-5494-416a-8a8d-47ff53f535bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.875083 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947b2e2-5494-416a-8a8d-47ff53f535bc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.898790 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1167/2338] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:55 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:55.903144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d203cc-4cc0-4c5d-bd86-70ac9fb31128 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.913576 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d203cc-4cc0-4c5d-bd86-70ac9fb31128 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.919034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7630a6f-e262-46e7-939f-afa973da2ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.919643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630a6f-e262-46e7-939f-afa973da2ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.919919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630a6f-e262-46e7-939f-afa973da2ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.920192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630a6f-e262-46e7-939f-afa973da2ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.920841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7630a6f-e262-46e7-939f-afa973da2ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.922484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7630a6f-e262-46e7-939f-afa973da2ce0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.933963 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1172/2339] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:55.938412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947b2e2-5494-416a-8a8d-47ff53f535bc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:55.949478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947b2e2-5494-416a-8a8d-47ff53f535bc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:55.954256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46c5a4-daea-489c-b661-33e84c463a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:55.954888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46c5a4-daea-489c-b661-33e84c463a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:55.955172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46c5a4-daea-489c-b661-33e84c463a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:55.955444 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46c5a4-daea-489c-b661-33e84c463a04 admin admin] PATH_INFO: `/v3/projects/74825084c4174676a1c837ff36c220cb/users/00e284e32bda4679942d5fd4d84076ac/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:55.970927 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46c5a4-daea-489c-b661-33e84c463a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74825084c4174676a1c837ff36c220cb, user_id=00e284e32bda4679942d5fd4d84076ac, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:55.973948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46c5a4-daea-489c-b661-33e84c463a04 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:55.989428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1168/2340] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:55 2022] PUT /identity/v3/projects/74825084c4174676a1c837ff36c220cb/users/00e284e32bda4679942d5fd4d84076ac/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:55.993688 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7630a6f-e262-46e7-939f-afa973da2ce0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:56.004266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7630a6f-e262-46e7-939f-afa973da2ce0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:56.013470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:56.067207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628849e-b53f-4411-b7c6-bf7069be89d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:56.067825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628849e-b53f-4411-b7c6-bf7069be89d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:56.068100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628849e-b53f-4411-b7c6-bf7069be89d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:56.068338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628849e-b53f-4411-b7c6-bf7069be89d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:56.068979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7628849e-b53f-4411-b7c6-bf7069be89d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:56.070654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7628849e-b53f-4411-b7c6-bf7069be89d2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:56.081771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1173/2341] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:56.086205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46c5a4-daea-489c-b661-33e84c463a04 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:56.097538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46c5a4-daea-489c-b661-33e84c463a04 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:56.102479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:56.103211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:56.103549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:56.103897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] PATH_INFO: `/v3/projects/74825084c4174676a1c837ff36c220cb/users/00e284e32bda4679942d5fd4d84076ac/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:56.106277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74825084c4174676a1c837ff36c220cb, user_id=00e284e32bda4679942d5fd4d84076ac, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:56.109005 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:56.120076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1169/2342] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:56 2022] PUT /identity/v3/projects/74825084c4174676a1c837ff36c220cb/users/00e284e32bda4679942d5fd4d84076ac/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:56.130908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3ed3-b42c-4567-a9b6-1abcd42ac185 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:56.131195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3ed3-b42c-4567-a9b6-1abcd42ac185 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:56.131505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3ed3-b42c-4567-a9b6-1abcd42ac185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:56.159755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eb7c3ed3-b42c-4567-a9b6-1abcd42ac185 None None] MFA Rules not processed for user `00e284e32bda4679942d5fd4d84076ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:56.160934 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eb7c3ed3-b42c-4567-a9b6-1abcd42ac185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:56.230421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1174/2343] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:56.242232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:56.244176 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:56.306131 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7296-4e1a-4b91-8ced-3afdd8c00124 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:56.310871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67972f39-987f-4bcb-816c-b7efe64232b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:56.311474 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67972f39-987f-4bcb-816c-b7efe64232b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:56.311760 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67972f39-987f-4bcb-816c-b7efe64232b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:56.312021 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67972f39-987f-4bcb-816c-b7efe64232b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:56.321236 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67972f39-987f-4bcb-816c-b7efe64232b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:56.322648 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67972f39-987f-4bcb-816c-b7efe64232b2 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:56.332663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1170/2344] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:56.362658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c3ed3-b42c-4567-a9b6-1abcd42ac185 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:56.439932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c3ed3-b42c-4567-a9b6-1abcd42ac185 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:56.445697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:56.446363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:56.446640 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:56.446908 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] PATH_INFO: `/v3/users/72c569f3d5864d7e93c66ea4a39f295e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:56.448291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c569f3d5864d7e93c66ea4a39f295e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:56.450444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:56.495243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:56.507135 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c569f3d5864d7e93c66ea4a39f295e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:56.508050 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c569f3d5864d7e93c66ea4a39f295e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:56.523016 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c569f3d5864d7e93c66ea4a39f295e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:56.530749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c569f3d5864d7e93c66ea4a39f295e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:56.531406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c569f3d5864d7e93c66ea4a39f295e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:56.545310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1175/2345] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:56 2022] DELETE /identity/v3/users/72c569f3d5864d7e93c66ea4a39f295e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:56.549952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c731d-c5e7-4743-9c59-f2ba1819ecbb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:56.550309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c731d-c5e7-4743-9c59-f2ba1819ecbb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:56.550590 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c731d-c5e7-4743-9c59-f2ba1819ecbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:56.570565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-be6c731d-c5e7-4743-9c59-f2ba1819ecbb None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:56.571806 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be6c731d-c5e7-4743-9c59-f2ba1819ecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:56.635912 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1171/2346] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:56.646245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:56.647902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:56.710837 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e660bde-f635-4398-aa3d-5c583cc31c8b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:56.715986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fe757-c2fa-4667-84ee-4b0baa93b360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:56.716624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe757-c2fa-4667-84ee-4b0baa93b360 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:56.716990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe757-c2fa-4667-84ee-4b0baa93b360 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:56.717420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe757-c2fa-4667-84ee-4b0baa93b360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:56.728779 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fe757-c2fa-4667-84ee-4b0baa93b360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:56.730915 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fe757-c2fa-4667-84ee-4b0baa93b360 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:56.741243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1176/2347] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:57.035833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c731d-c5e7-4743-9c59-f2ba1819ecbb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:57.037307 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be6c731d-c5e7-4743-9c59-f2ba1819ecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:57.049855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c731d-c5e7-4743-9c59-f2ba1819ecbb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:57.055446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:57.056191 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:57.056566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:57.056897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] PATH_INFO: `/v3/projects/1fc6fdf5e9484a0c9076cb5bb7ea368e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:57.058426 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc6fdf5e9484a0c9076cb5bb7ea368e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:57.060465 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:57.073433 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:57.113870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc6fdf5e9484a0c9076cb5bb7ea368e', 'request_id': 'req-88493352-c6dd-40fc-8648-9e66a4342d30'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.121609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc6fdf5e9484a0c9076cb5bb7ea368e', 'request_id': 'req-88493352-c6dd-40fc-8648-9e66a4342d30'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.130471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc6fdf5e9484a0c9076cb5bb7ea368e', 'request_id': 'req-88493352-c6dd-40fc-8648-9e66a4342d30'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.141386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] The token cache is being invalidate because project 1fc6fdf5e9484a0c9076cb5bb7ea368e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:57.141686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.142346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.143813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1172/2348] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:57 2022] DELETE /identity/v3/projects/1fc6fdf5e9484a0c9076cb5bb7ea368e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:57.148212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fe757-c2fa-4667-84ee-4b0baa93b360 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:57.159436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fe757-c2fa-4667-84ee-4b0baa93b360 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:57.238922 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:57.239598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:57.239883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:57.240139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] PATH_INFO: `/v3/users/00e284e32bda4679942d5fd4d84076ac` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:57.241473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e284e32bda4679942d5fd4d84076ac)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:57.243643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:57.293807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e284e32bda4679942d5fd4d84076ac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.294682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e284e32bda4679942d5fd4d84076ac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.302514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:57.307945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e284e32bda4679942d5fd4d84076ac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.315827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e284e32bda4679942d5fd4d84076ac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.316781 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-296927a9-3838-4c01-b8d5-7846a1ec641b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e284e32bda4679942d5fd4d84076ac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.332111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1177/2349] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:57 2022] DELETE /identity/v3/users/00e284e32bda4679942d5fd4d84076ac => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:57.619483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:57.621136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:57.636189 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88493352-c6dd-40fc-8648-9e66a4342d30 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:57.698478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:57.699703 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:57.700002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:57.700395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] PATH_INFO: `/v3/projects/74825084c4174676a1c837ff36c220cb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:57.701895 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74825084c4174676a1c837ff36c220cb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:57.704401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:57.756099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74825084c4174676a1c837ff36c220cb', 'request_id': 'req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.762954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74825084c4174676a1c837ff36c220cb', 'request_id': 'req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.773544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74825084c4174676a1c837ff36c220cb', 'request_id': 'req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.792576 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] The token cache is being invalidate because project 74825084c4174676a1c837ff36c220cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:57.792861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.793540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ea0bdb45-4de2-4ce6-9e15-22de84d608b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:57.795100 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1173/2350] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:57 2022] DELETE /identity/v3/projects/74825084c4174676a1c837ff36c220cb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:57.799974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170f1cf-3d5b-4e8a-b2c9-2b3da6e8da2c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:57.800260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170f1cf-3d5b-4e8a-b2c9-2b3da6e8da2c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:57.800589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170f1cf-3d5b-4e8a-b2c9-2b3da6e8da2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:57.822579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4170f1cf-3d5b-4e8a-b2c9-2b3da6e8da2c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:57.824909 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4170f1cf-3d5b-4e8a-b2c9-2b3da6e8da2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:57.889095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1178/2351] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:57.897834 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb26a2-c56e-480e-a2e7-522e070704bd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:57.898084 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb26a2-c56e-480e-a2e7-522e070704bd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:57.898413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb26a2-c56e-480e-a2e7-522e070704bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:57.918467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cfbb26a2-c56e-480e-a2e7-522e070704bd None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:57.919665 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cfbb26a2-c56e-480e-a2e7-522e070704bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:57.937567 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1174/2352] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:57.938219 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170f1cf-3d5b-4e8a-b2c9-2b3da6e8da2c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:57.948160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170f1cf-3d5b-4e8a-b2c9-2b3da6e8da2c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:57.953772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:57.954308 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:57.954608 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:57.954873 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:57.955523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:37:57.955818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:57.957397 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:57.974103 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1179/2353] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:37:57.977952 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb26a2-c56e-480e-a2e7-522e070704bd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:57.987742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb26a2-c56e-480e-a2e7-522e070704bd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:57.992338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c1623-653c-4cc2-9ce3-a0635c11c09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:57.993052 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1623-653c-4cc2-9ce3-a0635c11c09a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:57.993329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1623-653c-4cc2-9ce3-a0635c11c09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:57.993595 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1623-653c-4cc2-9ce3-a0635c11c09a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:57.994265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c1623-653c-4cc2-9ce3-a0635c11c09a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:57.996332 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c1623-653c-4cc2-9ce3-a0635c11c09a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.026139 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1175/2354] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:57 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:58.028281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.038598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86bb62-a395-4712-ab36-88e8e98115dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.097416 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b1fa6-04b1-46bb-b541-3ec4c6dabe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.098029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b1fa6-04b1-46bb-b541-3ec4c6dabe78 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.098330 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b1fa6-04b1-46bb-b541-3ec4c6dabe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.098592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b1fa6-04b1-46bb-b541-3ec4c6dabe78 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.099229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b1fa6-04b1-46bb-b541-3ec4c6dabe78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.101194 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b1fa6-04b1-46bb-b541-3ec4c6dabe78 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.125446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1180/2355] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:58 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:58.129605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c1623-653c-4cc2-9ce3-a0635c11c09a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.135275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:58.141987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c1623-653c-4cc2-9ce3-a0635c11c09a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.148330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86054b32-6924-4e29-a406-17c478d7551d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.148940 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054b32-6924-4e29-a406-17c478d7551d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.149220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054b32-6924-4e29-a406-17c478d7551d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.149490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054b32-6924-4e29-a406-17c478d7551d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.150167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86054b32-6924-4e29-a406-17c478d7551d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.151775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86054b32-6924-4e29-a406-17c478d7551d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.162732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1176/2356] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:37:58.166877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b1fa6-04b1-46bb-b541-3ec4c6dabe78 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.177504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b1fa6-04b1-46bb-b541-3ec4c6dabe78 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.183075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea58ba3-3cb6-4735-b99b-24882649399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.184090 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58ba3-3cb6-4735-b99b-24882649399b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.184369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58ba3-3cb6-4735-b99b-24882649399b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.184634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58ba3-3cb6-4735-b99b-24882649399b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.185275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea58ba3-3cb6-4735-b99b-24882649399b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.186902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea58ba3-3cb6-4735-b99b-24882649399b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.198741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1181/2357] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:58.203225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86054b32-6924-4e29-a406-17c478d7551d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.213237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86054b32-6924-4e29-a406-17c478d7551d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.217875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3ae54-2fac-4e5b-a5d3-85cebe507866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.218553 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3ae54-2fac-4e5b-a5d3-85cebe507866 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.218809 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3ae54-2fac-4e5b-a5d3-85cebe507866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.219079 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3ae54-2fac-4e5b-a5d3-85cebe507866 admin admin] PATH_INFO: `/v3/projects/3456b56ee49a45659c9887301187059c/users/1e754cf7a2a34c1d8f34924db013a47b/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.235707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3ae54-2fac-4e5b-a5d3-85cebe507866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3456b56ee49a45659c9887301187059c, user_id=1e754cf7a2a34c1d8f34924db013a47b, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.238557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3ae54-2fac-4e5b-a5d3-85cebe507866 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.252483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1177/2358] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:58 2022] PUT /identity/v3/projects/3456b56ee49a45659c9887301187059c/users/1e754cf7a2a34c1d8f34924db013a47b/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:58.266746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f63e3f-a5d9-4255-8f53-4ebc1b4e8d88 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.267030 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f63e3f-a5d9-4255-8f53-4ebc1b4e8d88 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.267300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f63e3f-a5d9-4255-8f53-4ebc1b4e8d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.297626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-45f63e3f-a5d9-4255-8f53-4ebc1b4e8d88 None None] MFA Rules not processed for user `1e754cf7a2a34c1d8f34924db013a47b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:58.298891 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-45f63e3f-a5d9-4255-8f53-4ebc1b4e8d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:58.336000 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:58.374167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1182/2359] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:58.378719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3ae54-2fac-4e5b-a5d3-85cebe507866 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.444786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3ae54-2fac-4e5b-a5d3-85cebe507866 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.449872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5a9f0-f45e-4977-9877-2a4fdc1e26e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.450522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5a9f0-f45e-4977-9877-2a4fdc1e26e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.450798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5a9f0-f45e-4977-9877-2a4fdc1e26e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.451056 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5a9f0-f45e-4977-9877-2a4fdc1e26e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.451726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5a9f0-f45e-4977-9877-2a4fdc1e26e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.453620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5a9f0-f45e-4977-9877-2a4fdc1e26e6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.485352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1178/2360] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:37:58 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:58.486625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f63e3f-a5d9-4255-8f53-4ebc1b4e8d88 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.498208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f63e3f-a5d9-4255-8f53-4ebc1b4e8d88 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.557642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4546cf-7746-4905-bdce-ebf29014527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.558386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4546cf-7746-4905-bdce-ebf29014527e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.558666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4546cf-7746-4905-bdce-ebf29014527e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.558937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4546cf-7746-4905-bdce-ebf29014527e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.559607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4546cf-7746-4905-bdce-ebf29014527e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.562223 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4546cf-7746-4905-bdce-ebf29014527e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.588988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1183/2361] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:37:58 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:37:58.592578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5a9f0-f45e-4977-9877-2a4fdc1e26e6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.603792 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5a9f0-f45e-4977-9877-2a4fdc1e26e6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.608739 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabcdbe-92a8-4ccb-b6c0-87b0914a69da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.609342 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabcdbe-92a8-4ccb-b6c0-87b0914a69da admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.609621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabcdbe-92a8-4ccb-b6c0-87b0914a69da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.609914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabcdbe-92a8-4ccb-b6c0-87b0914a69da admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.610610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabcdbe-92a8-4ccb-b6c0-87b0914a69da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.612215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabcdbe-92a8-4ccb-b6c0-87b0914a69da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.624098 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1179/2362] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:58.628113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4546cf-7746-4905-bdce-ebf29014527e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.639815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4546cf-7746-4905-bdce-ebf29014527e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.644556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e4132-7281-4da5-9bc9-8c356515e2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.645290 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e4132-7281-4da5-9bc9-8c356515e2ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.645633 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e4132-7281-4da5-9bc9-8c356515e2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.645964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e4132-7281-4da5-9bc9-8c356515e2ee admin admin] PATH_INFO: `/v3/projects/a0e3a5a1e2b3469f85c73ff7e9425242/users/93ecedc7808c481ab905b7b57ce44f5f/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.662027 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e4132-7281-4da5-9bc9-8c356515e2ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e3a5a1e2b3469f85c73ff7e9425242, user_id=93ecedc7808c481ab905b7b57ce44f5f, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.664731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e4132-7281-4da5-9bc9-8c356515e2ee admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.679614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1184/2363] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:58 2022] PUT /identity/v3/projects/a0e3a5a1e2b3469f85c73ff7e9425242/users/93ecedc7808c481ab905b7b57ce44f5f/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:58.684012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabcdbe-92a8-4ccb-b6c0-87b0914a69da admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.694004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabcdbe-92a8-4ccb-b6c0-87b0914a69da admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.755111 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003b9ce-918d-4ee0-ad19-d35f00c900f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.755724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003b9ce-918d-4ee0-ad19-d35f00c900f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.756002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003b9ce-918d-4ee0-ad19-d35f00c900f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.756297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003b9ce-918d-4ee0-ad19-d35f00c900f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.756962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003b9ce-918d-4ee0-ad19-d35f00c900f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.758594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003b9ce-918d-4ee0-ad19-d35f00c900f1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.772229 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1180/2364] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:37:58.773863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e4132-7281-4da5-9bc9-8c356515e2ee admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.784848 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e4132-7281-4da5-9bc9-8c356515e2ee admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:58.790356 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:58.791011 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.791315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.791574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] PATH_INFO: `/v3/projects/a0e3a5a1e2b3469f85c73ff7e9425242/users/93ecedc7808c481ab905b7b57ce44f5f/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.793758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e3a5a1e2b3469f85c73ff7e9425242, user_id=93ecedc7808c481ab905b7b57ce44f5f, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:58.796423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:58.807381 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1185/2365] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:37:58 2022] PUT /identity/v3/projects/a0e3a5a1e2b3469f85c73ff7e9425242/users/93ecedc7808c481ab905b7b57ce44f5f/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:58.817759 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c12ba-1a84-4974-a06f-4352a02798a3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:58.818045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c12ba-1a84-4974-a06f-4352a02798a3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:58.818385 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c12ba-1a84-4974-a06f-4352a02798a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:58.846913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9f4c12ba-1a84-4974-a06f-4352a02798a3 None None] MFA Rules not processed for user `93ecedc7808c481ab905b7b57ce44f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:58.848126 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9f4c12ba-1a84-4974-a06f-4352a02798a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:58.874528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:58.941886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1181/2366] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:58 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:58.954488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:58.956532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:59.023485 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717790e4-020a-41b3-947c-26ab3a8c9047 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:59.027180 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90385a-c346-49e5-8964-e8c87208e76b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:59.027809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90385a-c346-49e5-8964-e8c87208e76b service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:59.028084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90385a-c346-49e5-8964-e8c87208e76b service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:59.028373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90385a-c346-49e5-8964-e8c87208e76b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:59.037662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90385a-c346-49e5-8964-e8c87208e76b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:59.039140 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90385a-c346-49e5-8964-e8c87208e76b service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:59.048400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1186/2367] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:58 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:59.189839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c12ba-1a84-4974-a06f-4352a02798a3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:59.256013 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c12ba-1a84-4974-a06f-4352a02798a3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:59.260712 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:59.261464 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:59.261796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:59.262153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] PATH_INFO: `/v3/users/1e754cf7a2a34c1d8f34924db013a47b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:59.263344 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e754cf7a2a34c1d8f34924db013a47b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:59.265256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:59.312518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e754cf7a2a34c1d8f34924db013a47b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.313418 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e754cf7a2a34c1d8f34924db013a47b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.330707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e754cf7a2a34c1d8f34924db013a47b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.338145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e754cf7a2a34c1d8f34924db013a47b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.338937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e754cf7a2a34c1d8f34924db013a47b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.353846 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1182/2368] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:59 2022] DELETE /identity/v3/users/1e754cf7a2a34c1d8f34924db013a47b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:37:59.358444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266c37b-eeef-45e2-9815-de06b958031a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:59.358939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266c37b-eeef-45e2-9815-de06b958031a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:59.359036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266c37b-eeef-45e2-9815-de06b958031a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:59.379950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e266c37b-eeef-45e2-9815-de06b958031a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:59.381503 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e266c37b-eeef-45e2-9815-de06b958031a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:59.414003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:59.456876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1187/2369] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:59.467508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:59.468999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:59.519033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:59.535341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b97d03-e943-42e2-93a9-1ba29b744466 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:59.541368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd36d07-6dff-4ae8-a90f-5eb0307f1a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:59.542149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd36d07-6dff-4ae8-a90f-5eb0307f1a33 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:59.542727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd36d07-6dff-4ae8-a90f-5eb0307f1a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:59.542926 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd36d07-6dff-4ae8-a90f-5eb0307f1a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:59.553117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd36d07-6dff-4ae8-a90f-5eb0307f1a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:59.554605 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd36d07-6dff-4ae8-a90f-5eb0307f1a33 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:59.565950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1183/2370] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:37:59.866229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266c37b-eeef-45e2-9815-de06b958031a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:59.867706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e266c37b-eeef-45e2-9815-de06b958031a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:59.880914 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266c37b-eeef-45e2-9815-de06b958031a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:59.894857 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:59.895656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:59.895872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:59.896060 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] PATH_INFO: `/v3/projects/3456b56ee49a45659c9887301187059c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:59.897343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3456b56ee49a45659c9887301187059c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:59.902889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:59.953877 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3456b56ee49a45659c9887301187059c', 'request_id': 'req-9c9d3bfe-5a6c-422d-b790-ea97a413c831'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.964400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3456b56ee49a45659c9887301187059c', 'request_id': 'req-9c9d3bfe-5a6c-422d-b790-ea97a413c831'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.972028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3456b56ee49a45659c9887301187059c', 'request_id': 'req-9c9d3bfe-5a6c-422d-b790-ea97a413c831'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.984299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] The token cache is being invalidate because project 3456b56ee49a45659c9887301187059c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:37:59.984599 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.985296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:37:59.987062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1188/2371] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:37:59 2022] DELETE /identity/v3/projects/3456b56ee49a45659c9887301187059c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:37:59.990993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd36d07-6dff-4ae8-a90f-5eb0307f1a33 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:00.002000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd36d07-6dff-4ae8-a90f-5eb0307f1a33 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:00.076682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:00.077442 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:00.077708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:00.078004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] PATH_INFO: `/v3/users/93ecedc7808c481ab905b7b57ce44f5f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:00.079297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ecedc7808c481ab905b7b57ce44f5f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:00.081361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:00.131116 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ecedc7808c481ab905b7b57ce44f5f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.132058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ecedc7808c481ab905b7b57ce44f5f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.147363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ecedc7808c481ab905b7b57ce44f5f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.154206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ecedc7808c481ab905b7b57ce44f5f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.154939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1acfc4fd-62ed-494c-9a34-fcc220f3a329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ecedc7808c481ab905b7b57ce44f5f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.170187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1184/2372] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:37:59 2022] DELETE /identity/v3/users/93ecedc7808c481ab905b7b57ce44f5f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:00.485701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:00.487178 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:00.530024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d3bfe-5a6c-422d-b790-ea97a413c831 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:00.595883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:00.597115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:00.597496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:00.597774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] PATH_INFO: `/v3/projects/a0e3a5a1e2b3469f85c73ff7e9425242` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:00.601036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e3a5a1e2b3469f85c73ff7e9425242)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:00.603157 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:00.665884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e3a5a1e2b3469f85c73ff7e9425242', 'request_id': 'req-19c3dbd2-3915-4af7-89eb-5cb158a76288'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.668700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:00.675765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e3a5a1e2b3469f85c73ff7e9425242', 'request_id': 'req-19c3dbd2-3915-4af7-89eb-5cb158a76288'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.683969 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e3a5a1e2b3469f85c73ff7e9425242', 'request_id': 'req-19c3dbd2-3915-4af7-89eb-5cb158a76288'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.695012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] The token cache is being invalidate because project a0e3a5a1e2b3469f85c73ff7e9425242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:00.695299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.695974 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-19c3dbd2-3915-4af7-89eb-5cb158a76288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:00.697468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1189/2373] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:00 2022] DELETE /identity/v3/projects/a0e3a5a1e2b3469f85c73ff7e9425242 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:00.702949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19590af-f015-4524-a7c5-c24f8eefa4b0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:00.703631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19590af-f015-4524-a7c5-c24f8eefa4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:00.703631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19590af-f015-4524-a7c5-c24f8eefa4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:00.724558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a19590af-f015-4524-a7c5-c24f8eefa4b0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:00.725899 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a19590af-f015-4524-a7c5-c24f8eefa4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:00.793652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1185/2374] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:38:00.802625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044e66-29c6-4ae0-9204-90b0aec8b464 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:00.803006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044e66-29c6-4ae0-9204-90b0aec8b464 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:00.803420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044e66-29c6-4ae0-9204-90b0aec8b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:00.825567 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-11044e66-29c6-4ae0-9204-90b0aec8b464 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:00.826954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-11044e66-29c6-4ae0-9204-90b0aec8b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:00.837143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1190/2375] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:00.840996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19590af-f015-4524-a7c5-c24f8eefa4b0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:00.848239 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:00.853623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19590af-f015-4524-a7c5-c24f8eefa4b0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:00.860932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:00.862142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:00.862288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:00.863323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:00.863323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:38:00.863714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:00.865752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:00.884309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1186/2376] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:38:00.889110 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11044e66-29c6-4ae0-9204-90b0aec8b464 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:00.905605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11044e66-29c6-4ae0-9204-90b0aec8b464 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:00.925534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328ab6-2643-4a9f-86a9-6c6004b3c066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:00.926328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328ab6-2643-4a9f-86a9-6c6004b3c066 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:00.926708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328ab6-2643-4a9f-86a9-6c6004b3c066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:00.927173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328ab6-2643-4a9f-86a9-6c6004b3c066 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:00.930946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3328ab6-2643-4a9f-86a9-6c6004b3c066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:00.933175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3328ab6-2643-4a9f-86a9-6c6004b3c066 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:00.986594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1191/2377] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:00 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:38:00.988170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:01.008232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02155efd-2a28-4a91-a843-98f18a8c5711 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:01.089947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59014837-a39b-4db8-8acb-73eea63967a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:01.090682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59014837-a39b-4db8-8acb-73eea63967a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:01.091121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59014837-a39b-4db8-8acb-73eea63967a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:01.091337 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59014837-a39b-4db8-8acb-73eea63967a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:01.092349 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59014837-a39b-4db8-8acb-73eea63967a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:01.095197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59014837-a39b-4db8-8acb-73eea63967a8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:01.135115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328ab6-2643-4a9f-86a9-6c6004b3c066 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:01.135685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1187/2378] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:00 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:01.147944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328ab6-2643-4a9f-86a9-6c6004b3c066 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:01.152975 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63c3f2-e537-49af-a2bd-6b1b43f920c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:01.153623 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63c3f2-e537-49af-a2bd-6b1b43f920c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:01.153936 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63c3f2-e537-49af-a2bd-6b1b43f920c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:01.154289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63c3f2-e537-49af-a2bd-6b1b43f920c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:01.154976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63c3f2-e537-49af-a2bd-6b1b43f920c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:01.156622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63c3f2-e537-49af-a2bd-6b1b43f920c2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:01.167689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1192/2379] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:01.172080 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59014837-a39b-4db8-8acb-73eea63967a8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:01.183115 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59014837-a39b-4db8-8acb-73eea63967a8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:01.188416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dbeaa-43ff-4f42-b957-c1a9a94bc827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:01.189119 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dbeaa-43ff-4f42-b957-c1a9a94bc827 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:01.189469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dbeaa-43ff-4f42-b957-c1a9a94bc827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:01.189750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dbeaa-43ff-4f42-b957-c1a9a94bc827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:01.190441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dbeaa-43ff-4f42-b957-c1a9a94bc827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:01.192059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dbeaa-43ff-4f42-b957-c1a9a94bc827 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:01.203574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1188/2380] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:01.207796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63c3f2-e537-49af-a2bd-6b1b43f920c2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:01.219164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63c3f2-e537-49af-a2bd-6b1b43f920c2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:01.224224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:01.225065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:01.225316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:01.225680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] PATH_INFO: `/v3/projects/7bb98db6ee62489a9bfb438dcc2feef5/users/8f65dbb7cd6343a3aeab4f9ef20fe57e/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:01.241970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb98db6ee62489a9bfb438dcc2feef5, user_id=8f65dbb7cd6343a3aeab4f9ef20fe57e, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:01.244726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:01.260211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1193/2381] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:01 2022] PUT /identity/v3/projects/7bb98db6ee62489a9bfb438dcc2feef5/users/8f65dbb7cd6343a3aeab4f9ef20fe57e/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:01.271612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeca39-f4a9-4a02-912e-e2b9cb5c3c18 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:01.271897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeca39-f4a9-4a02-912e-e2b9cb5c3c18 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:01.272171 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeca39-f4a9-4a02-912e-e2b9cb5c3c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:01.304407 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3fbeca39-f4a9-4a02-912e-e2b9cb5c3c18 None None] MFA Rules not processed for user `8f65dbb7cd6343a3aeab4f9ef20fe57e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:01.305624 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fbeca39-f4a9-4a02-912e-e2b9cb5c3c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:01.378503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1189/2382] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:01.390216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:01.392143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:01.454551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6a824-37cf-4a06-835c-b0a8eba52f79 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:01.459007 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:01.460079 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:01.460352 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:01.460634 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:01.470940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:01.472757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:01.482923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1194/2383] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:10.208042 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbeca39-f4a9-4a02-912e-e2b9cb5c3c18 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:10.209620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fbeca39-f4a9-4a02-912e-e2b9cb5c3c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:10.270313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbeca39-f4a9-4a02-912e-e2b9cb5c3c18 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:10.274731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:10.275331 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:10.275600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:10.275853 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:10.277932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:10.291977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:10.345653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:10.347482 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:10.357741 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1190/2384] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:11.478058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=3,keystone:SELECT=59,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:13.732926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:13.734589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.752412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8db238-423b-4bce-86d0-07fb0bce9b58 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:13.757584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:13.758292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.758572 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.758818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] PATH_INFO: `/v3/users/006ae080f2884719bd4ddabe815d0d01` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.774145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=006ae080f2884719bd4ddabe815d0d01)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:13.776091 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:13.829809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006ae080f2884719bd4ddabe815d0d01'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.830645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006ae080f2884719bd4ddabe815d0d01'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.845309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006ae080f2884719bd4ddabe815d0d01'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.861654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006ae080f2884719bd4ddabe815d0d01'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.862526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-31f1a038-baa8-4e88-b527-6a12be6283f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006ae080f2884719bd4ddabe815d0d01'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:13.880630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1195/2385] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:13 2022] DELETE /identity/v3/users/006ae080f2884719bd4ddabe815d0d01 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:14.073735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.097112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc97dc-e9a6-477d-8adb-fa00397c1519 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.165025 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.165559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.165841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.166140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] PATH_INFO: `/v3/projects/80e2c88e09a546bdbe9e644295a237bc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.177398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e2c88e09a546bdbe9e644295a237bc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.179351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.227766 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e2c88e09a546bdbe9e644295a237bc', 'request_id': 'req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:14.234404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e2c88e09a546bdbe9e644295a237bc', 'request_id': 'req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:14.242879 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e2c88e09a546bdbe9e644295a237bc', 'request_id': 'req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:14.255007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] The token cache is being invalidate because project 80e2c88e09a546bdbe9e644295a237bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:14.255394 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:14.255942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-253e2d6c-a5eb-4c4e-9fd8-4767a8d4a843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:14.257743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1191/2386] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:14 2022] DELETE /identity/v3/projects/80e2c88e09a546bdbe9e644295a237bc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:14.262905 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c40c6-3830-4fec-8e7f-767a584cab04 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.263266 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c40c6-3830-4fec-8e7f-767a584cab04 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.263622 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c40c6-3830-4fec-8e7f-767a584cab04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.284794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-559c40c6-3830-4fec-8e7f-767a584cab04 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:14.285994 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-559c40c6-3830-4fec-8e7f-767a584cab04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:14.347557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1196/2387] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:14.355956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385b3ac-257b-4bca-91f0-3351c97141c0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.356250 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385b3ac-257b-4bca-91f0-3351c97141c0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.356523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385b3ac-257b-4bca-91f0-3351c97141c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.378621 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6385b3ac-257b-4bca-91f0-3351c97141c0 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:14.379849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6385b3ac-257b-4bca-91f0-3351c97141c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:14.389738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1192/2388] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:14.393741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c40c6-3830-4fec-8e7f-767a584cab04 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.404119 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c40c6-3830-4fec-8e7f-767a584cab04 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.409352 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.410444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.410880 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.411379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.412540 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:38:14.413065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.416598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.432015 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1197/2389] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:38:14.436780 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385b3ac-257b-4bca-91f0-3351c97141c0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.454913 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385b3ac-257b-4bca-91f0-3351c97141c0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.460889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a8057-100f-4625-a71d-ccb39f35beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.461524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8057-100f-4625-a71d-ccb39f35beec admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.461799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8057-100f-4625-a71d-ccb39f35beec admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.462067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8057-100f-4625-a71d-ccb39f35beec admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.462737 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a8057-100f-4625-a71d-ccb39f35beec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.465425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a8057-100f-4625-a71d-ccb39f35beec admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.506937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1193/2390] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:14 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:14.511749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.523084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad84f3-8dda-4f1d-b6b7-7441d251e913 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.574484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:14.591058 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1c00-5948-43a6-8b3e-836da15e299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.591677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1c00-5948-43a6-8b3e-836da15e299f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.591954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1c00-5948-43a6-8b3e-836da15e299f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.592229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1c00-5948-43a6-8b3e-836da15e299f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.592893 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b1c00-5948-43a6-8b3e-836da15e299f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.594946 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b1c00-5948-43a6-8b3e-836da15e299f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.622726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1198/2391] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:14 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:14.627167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a8057-100f-4625-a71d-ccb39f35beec admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.632970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:14.641179 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a8057-100f-4625-a71d-ccb39f35beec admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.650884 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa189a54-96a8-45d2-a054-9e9589b4f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.651514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa189a54-96a8-45d2-a054-9e9589b4f872 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.651801 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa189a54-96a8-45d2-a054-9e9589b4f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.652085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa189a54-96a8-45d2-a054-9e9589b4f872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.652954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa189a54-96a8-45d2-a054-9e9589b4f872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.654860 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa189a54-96a8-45d2-a054-9e9589b4f872 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.666210 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1194/2392] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:38:14.670329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1c00-5948-43a6-8b3e-836da15e299f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.681731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1c00-5948-43a6-8b3e-836da15e299f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.686692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29bb36-0476-4a6f-9a02-dcd2c2ee5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.687289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29bb36-0476-4a6f-9a02-dcd2c2ee5fab admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.687562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29bb36-0476-4a6f-9a02-dcd2c2ee5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.687927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29bb36-0476-4a6f-9a02-dcd2c2ee5fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.688793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29bb36-0476-4a6f-9a02-dcd2c2ee5fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.690478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29bb36-0476-4a6f-9a02-dcd2c2ee5fab admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.701773 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1199/2393] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:14.707586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa189a54-96a8-45d2-a054-9e9589b4f872 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.717819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa189a54-96a8-45d2-a054-9e9589b4f872 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.723882 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be4386-f172-48a1-963e-e9d9ac03c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.724626 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be4386-f172-48a1-963e-e9d9ac03c942 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.724855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be4386-f172-48a1-963e-e9d9ac03c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.725144 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be4386-f172-48a1-963e-e9d9ac03c942 admin admin] PATH_INFO: `/v3/projects/d2b7adf583234739805f744c8776454c/users/563da948f14845638e8e44ede0a6f7aa/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.741812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be4386-f172-48a1-963e-e9d9ac03c942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b7adf583234739805f744c8776454c, user_id=563da948f14845638e8e44ede0a6f7aa, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.744494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be4386-f172-48a1-963e-e9d9ac03c942 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.759443 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1195/2394] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:14 2022] PUT /identity/v3/projects/d2b7adf583234739805f744c8776454c/users/563da948f14845638e8e44ede0a6f7aa/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:14.771189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b775-872d-45ec-b8cc-0ae91712c083 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.771685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b775-872d-45ec-b8cc-0ae91712c083 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.771797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b775-872d-45ec-b8cc-0ae91712c083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.799954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6771b775-872d-45ec-b8cc-0ae91712c083 None None] MFA Rules not processed for user `563da948f14845638e8e44ede0a6f7aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:14.801170 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6771b775-872d-45ec-b8cc-0ae91712c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:14.872639 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1200/2395] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:14.877268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be4386-f172-48a1-963e-e9d9ac03c942 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:14.948188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be4386-f172-48a1-963e-e9d9ac03c942 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:14.953821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0e88-00c2-49b0-9dbc-39123ed60e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:14.954524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0e88-00c2-49b0-9dbc-39123ed60e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:14.954772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0e88-00c2-49b0-9dbc-39123ed60e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:14.955036 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0e88-00c2-49b0-9dbc-39123ed60e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:14.955663 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0e88-00c2-49b0-9dbc-39123ed60e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:14.957554 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0e88-00c2-49b0-9dbc-39123ed60e76 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:14.985995 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1196/2396] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:14 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:14.991672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771b775-872d-45ec-b8cc-0ae91712c083 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:15.003117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771b775-872d-45ec-b8cc-0ae91712c083 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:15.064036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7d598-e578-4c6f-b343-615fb8c67f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:15.064642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7d598-e578-4c6f-b343-615fb8c67f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:15.064919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7d598-e578-4c6f-b343-615fb8c67f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:15.065190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7d598-e578-4c6f-b343-615fb8c67f15 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:15.065849 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7d598-e578-4c6f-b343-615fb8c67f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:15.067827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7d598-e578-4c6f-b343-615fb8c67f15 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:15.091168 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1201/2397] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:14 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:38:15.095418 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0e88-00c2-49b0-9dbc-39123ed60e76 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:15.106522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0e88-00c2-49b0-9dbc-39123ed60e76 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:15.111607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303be24-890e-4db3-b625-eebb70d5e8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:15.112201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303be24-890e-4db3-b625-eebb70d5e8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:15.112483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303be24-890e-4db3-b625-eebb70d5e8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:15.112755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303be24-890e-4db3-b625-eebb70d5e8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:15.113395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303be24-890e-4db3-b625-eebb70d5e8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:15.115048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303be24-890e-4db3-b625-eebb70d5e8b4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:15.125753 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1197/2398] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:15.130148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7d598-e578-4c6f-b343-615fb8c67f15 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:15.140278 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7d598-e578-4c6f-b343-615fb8c67f15 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:15.145654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c50bc-8d6c-414c-95bd-e6d4f92abfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:15.146490 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c50bc-8d6c-414c-95bd-e6d4f92abfbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:15.146907 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c50bc-8d6c-414c-95bd-e6d4f92abfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:15.147189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c50bc-8d6c-414c-95bd-e6d4f92abfbb admin admin] PATH_INFO: `/v3/projects/ebacec42aed94a03915f8899e191c450/users/18e0238d41fa412da51bc62f640cc257/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:15.163105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c50bc-8d6c-414c-95bd-e6d4f92abfbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebacec42aed94a03915f8899e191c450, user_id=18e0238d41fa412da51bc62f640cc257, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:15.165736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c50bc-8d6c-414c-95bd-e6d4f92abfbb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:15.180860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1202/2399] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:15 2022] PUT /identity/v3/projects/ebacec42aed94a03915f8899e191c450/users/18e0238d41fa412da51bc62f640cc257/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:15.185058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303be24-890e-4db3-b625-eebb70d5e8b4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:15.195117 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303be24-890e-4db3-b625-eebb70d5e8b4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:15.254151 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051882d-732a-457a-907b-7f0b24af1c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:15.254670 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051882d-732a-457a-907b-7f0b24af1c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:15.254939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051882d-732a-457a-907b-7f0b24af1c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:15.255208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051882d-732a-457a-907b-7f0b24af1c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:15.255841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051882d-732a-457a-907b-7f0b24af1c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:15.257404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051882d-732a-457a-907b-7f0b24af1c6f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:15.267702 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1198/2400] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:15.273745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c50bc-8d6c-414c-95bd-e6d4f92abfbb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:15.284770 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c50bc-8d6c-414c-95bd-e6d4f92abfbb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:15.290906 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:15.291571 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:15.291850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:15.292162 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] PATH_INFO: `/v3/projects/ebacec42aed94a03915f8899e191c450/users/18e0238d41fa412da51bc62f640cc257/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:15.294602 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebacec42aed94a03915f8899e191c450, user_id=18e0238d41fa412da51bc62f640cc257, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:15.297265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:15.308808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1203/2401] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:15 2022] PUT /identity/v3/projects/ebacec42aed94a03915f8899e191c450/users/18e0238d41fa412da51bc62f640cc257/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:15.319659 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add595-4acb-4c1a-9040-30d1c0b3b3ca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:15.319940 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add595-4acb-4c1a-9040-30d1c0b3b3ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:15.320208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add595-4acb-4c1a-9040-30d1c0b3b3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:15.351447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-26add595-4acb-4c1a-9040-30d1c0b3b3ca None None] MFA Rules not processed for user `18e0238d41fa412da51bc62f640cc257`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:15.352711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-26add595-4acb-4c1a-9040-30d1c0b3b3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:15.380147 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:15.425215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1199/2402] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:38:15.436730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:15.438742 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:15.450555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:15.501658 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770b1ea-417a-4ebd-9277-d34b1cf62854 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:15.506181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b45b5-7329-4417-9758-f2c8e7cfafd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:15.506647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b45b5-7329-4417-9758-f2c8e7cfafd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:15.506926 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b45b5-7329-4417-9758-f2c8e7cfafd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:15.507192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b45b5-7329-4417-9758-f2c8e7cfafd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:15.517413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b45b5-7329-4417-9758-f2c8e7cfafd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:15.518918 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b45b5-7329-4417-9758-f2c8e7cfafd1 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:15.528879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1204/2403] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:15.613486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26add595-4acb-4c1a-9040-30d1c0b3b3ca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:15.623977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26add595-4acb-4c1a-9040-30d1c0b3b3ca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:15.628359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e59e7-337a-4f1a-974a-92b2100e5f01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:15.628963 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e59e7-337a-4f1a-974a-92b2100e5f01 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:15.629237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e59e7-337a-4f1a-974a-92b2100e5f01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:15.629503 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e59e7-337a-4f1a-974a-92b2100e5f01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:15.692399 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e59e7-337a-4f1a-974a-92b2100e5f01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:15.693851 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e59e7-337a-4f1a-974a-92b2100e5f01 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:15.706711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1200/2404] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:25.524243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=22 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:25.699591 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=45 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:32.139729 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b45b5-7329-4417-9758-f2c8e7cfafd1 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:32.208661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b45b5-7329-4417-9758-f2c8e7cfafd1 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:32.213568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:32.214297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:32.214649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:32.215109 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] PATH_INFO: `/v3/users/563da948f14845638e8e44ede0a6f7aa` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:32.216333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563da948f14845638e8e44ede0a6f7aa)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:32.218276 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:32.269343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563da948f14845638e8e44ede0a6f7aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.269762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563da948f14845638e8e44ede0a6f7aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.281799 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563da948f14845638e8e44ede0a6f7aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.289087 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563da948f14845638e8e44ede0a6f7aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.289820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563da948f14845638e8e44ede0a6f7aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.304057 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1205/2405] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:32 2022] DELETE /identity/v3/users/563da948f14845638e8e44ede0a6f7aa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:32.308548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a9308-c458-4287-bc9b-3ae16e7983ce None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:32.309087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a9308-c458-4287-bc9b-3ae16e7983ce None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:32.309087 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a9308-c458-4287-bc9b-3ae16e7983ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:32.330586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-060a9308-c458-4287-bc9b-3ae16e7983ce None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:32.331788 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-060a9308-c458-4287-bc9b-3ae16e7983ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:32.393707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1201/2406] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:32.404144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:32.405617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:32.465463 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796748c6-72b2-4f9f-b094-f45e7b88fa32 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:32.469367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe7312-b62e-42d0-9d36-d58ac929eeb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:32.469980 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe7312-b62e-42d0-9d36-d58ac929eeb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:32.470285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe7312-b62e-42d0-9d36-d58ac929eeb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:32.470560 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe7312-b62e-42d0-9d36-d58ac929eeb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:32.481855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe7312-b62e-42d0-9d36-d58ac929eeb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:32.483417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe7312-b62e-42d0-9d36-d58ac929eeb9 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:32.498513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1206/2407] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:32.776221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a9308-c458-4287-bc9b-3ae16e7983ce None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:32.777987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-060a9308-c458-4287-bc9b-3ae16e7983ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:32.791772 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a9308-c458-4287-bc9b-3ae16e7983ce None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:32.797612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:32.798401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:32.798620 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:32.798872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] PATH_INFO: `/v3/projects/d2b7adf583234739805f744c8776454c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:32.800204 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b7adf583234739805f744c8776454c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:32.802045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:32.853830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b7adf583234739805f744c8776454c', 'request_id': 'req-c6fe7124-8871-44a5-a1a5-c6a664973fc8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.860172 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b7adf583234739805f744c8776454c', 'request_id': 'req-c6fe7124-8871-44a5-a1a5-c6a664973fc8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.869549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b7adf583234739805f744c8776454c', 'request_id': 'req-c6fe7124-8871-44a5-a1a5-c6a664973fc8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.880277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] The token cache is being invalidate because project d2b7adf583234739805f744c8776454c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:32.880566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.881295 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:32.883280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1202/2408] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:32 2022] DELETE /identity/v3/projects/d2b7adf583234739805f744c8776454c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:32.888035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe7312-b62e-42d0-9d36-d58ac929eeb9 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:32.900191 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe7312-b62e-42d0-9d36-d58ac929eeb9 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:32.921286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:32.962507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:32.963126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:32.963398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:32.963684 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] PATH_INFO: `/v3/users/18e0238d41fa412da51bc62f640cc257` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:32.964802 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e0238d41fa412da51bc62f640cc257)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:32.966772 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:33.018251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e0238d41fa412da51bc62f640cc257'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.018874 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e0238d41fa412da51bc62f640cc257'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.031028 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e0238d41fa412da51bc62f640cc257'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.038572 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e0238d41fa412da51bc62f640cc257'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.039282 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c62c4ef-b710-4344-9a1d-6993249d3e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e0238d41fa412da51bc62f640cc257'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.053263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1207/2409] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:32 2022] DELETE /identity/v3/users/18e0238d41fa412da51bc62f640cc257 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:33.343034 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:33.344631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:33.361761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe7124-8871-44a5-a1a5-c6a664973fc8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:33.424282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:33.424834 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.425140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.425369 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] PATH_INFO: `/v3/projects/ebacec42aed94a03915f8899e191c450` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.426522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebacec42aed94a03915f8899e191c450)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:33.428493 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:33.459928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:33.479478 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebacec42aed94a03915f8899e191c450', 'request_id': 'req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.485750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebacec42aed94a03915f8899e191c450', 'request_id': 'req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.495054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebacec42aed94a03915f8899e191c450', 'request_id': 'req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.506218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] The token cache is being invalidate because project ebacec42aed94a03915f8899e191c450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:33.506502 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.507430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7e7ca4fa-3b21-4c07-aaf6-3c3dabe6cc37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:33.508922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1203/2410] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:33 2022] DELETE /identity/v3/projects/ebacec42aed94a03915f8899e191c450 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:33.516715 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ec3b-af02-4cdd-82b7-dd365b0b1bc9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.517208 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ec3b-af02-4cdd-82b7-dd365b0b1bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.517345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ec3b-af02-4cdd-82b7-dd365b0b1bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.541329 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9381ec3b-af02-4cdd-82b7-dd365b0b1bc9 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:33.542533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9381ec3b-af02-4cdd-82b7-dd365b0b1bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:33.601875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1208/2411] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:33.610202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bde6e1-d9a7-4b29-b8b5-86b927db60b3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.610521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bde6e1-d9a7-4b29-b8b5-86b927db60b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.610779 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bde6e1-d9a7-4b29-b8b5-86b927db60b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.630928 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-64bde6e1-d9a7-4b29-b8b5-86b927db60b3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:33.632212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-64bde6e1-d9a7-4b29-b8b5-86b927db60b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:33.640103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1204/2412] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:33.644552 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381ec3b-af02-4cdd-82b7-dd365b0b1bc9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:33.655153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381ec3b-af02-4cdd-82b7-dd365b0b1bc9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:33.660529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:33.661136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.661448 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.661721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.662401 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:38:33.662735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:33.664572 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:33.677412 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1209/2413] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:38:33.683633 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bde6e1-d9a7-4b29-b8b5-86b927db60b3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:33.694294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bde6e1-d9a7-4b29-b8b5-86b927db60b3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:33.699652 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3175be-e394-4641-8275-ed91dc071c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:33.700468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175be-e394-4641-8275-ed91dc071c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.700787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175be-e394-4641-8275-ed91dc071c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.701030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175be-e394-4641-8275-ed91dc071c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.701701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3175be-e394-4641-8275-ed91dc071c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:33.706372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3175be-e394-4641-8275-ed91dc071c98 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:33.739393 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1205/2414] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:33 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:33.744073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:33.755147 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e8332-24e3-4571-bf4d-ea42b22f5a42 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:33.786233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:33.818363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72159aa4-58f3-4882-86e0-c37de5135266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:33.818983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72159aa4-58f3-4882-86e0-c37de5135266 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.819380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72159aa4-58f3-4882-86e0-c37de5135266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.819551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72159aa4-58f3-4882-86e0-c37de5135266 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.820299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72159aa4-58f3-4882-86e0-c37de5135266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:33.822317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72159aa4-58f3-4882-86e0-c37de5135266 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:33.845379 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1210/2415] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:33 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:38:33.851252 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3175be-e394-4641-8275-ed91dc071c98 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:33.866850 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3175be-e394-4641-8275-ed91dc071c98 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:33.871578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84143c-bcc4-49d0-9882-2b9b11758e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:33.872196 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84143c-bcc4-49d0-9882-2b9b11758e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.872472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84143c-bcc4-49d0-9882-2b9b11758e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.872796 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84143c-bcc4-49d0-9882-2b9b11758e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.873439 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84143c-bcc4-49d0-9882-2b9b11758e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:33.875078 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84143c-bcc4-49d0-9882-2b9b11758e23 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:33.885082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1206/2416] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:33.889111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72159aa4-58f3-4882-86e0-c37de5135266 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:33.900100 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72159aa4-58f3-4882-86e0-c37de5135266 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:33.905780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a974b-2f8f-4694-81d2-55c72aab2788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:33.906415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a974b-2f8f-4694-81d2-55c72aab2788 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.906697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a974b-2f8f-4694-81d2-55c72aab2788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.906968 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a974b-2f8f-4694-81d2-55c72aab2788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.907627 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a974b-2f8f-4694-81d2-55c72aab2788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:33.909248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a974b-2f8f-4694-81d2-55c72aab2788 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:33.921338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1211/2417] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:33.925637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84143c-bcc4-49d0-9882-2b9b11758e23 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:33.936941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84143c-bcc4-49d0-9882-2b9b11758e23 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:33.942232 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d527c6-f200-482c-8227-29466c17df3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:33.942946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d527c6-f200-482c-8227-29466c17df3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.943220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d527c6-f200-482c-8227-29466c17df3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.943490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d527c6-f200-482c-8227-29466c17df3e admin admin] PATH_INFO: `/v3/projects/df4a8e78415b43bd9c4a1201e15c9155/users/74e077ae6b974b6ab029b53db85e9da7/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:33.959450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d527c6-f200-482c-8227-29466c17df3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4a8e78415b43bd9c4a1201e15c9155, user_id=74e077ae6b974b6ab029b53db85e9da7, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:33.962832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d527c6-f200-482c-8227-29466c17df3e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:33.978352 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1207/2418] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:33 2022] PUT /identity/v3/projects/df4a8e78415b43bd9c4a1201e15c9155/users/74e077ae6b974b6ab029b53db85e9da7/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:33.989182 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f8d5d-0c44-4de6-b27c-dff30e832fc7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:33.989421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f8d5d-0c44-4de6-b27c-dff30e832fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:33.989713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f8d5d-0c44-4de6-b27c-dff30e832fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.019344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ab9f8d5d-0c44-4de6-b27c-dff30e832fc7 None None] MFA Rules not processed for user `74e077ae6b974b6ab029b53db85e9da7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:34.020562 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab9f8d5d-0c44-4de6-b27c-dff30e832fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:34.100153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1212/2419] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:34.105145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d527c6-f200-482c-8227-29466c17df3e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:34.171059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d527c6-f200-482c-8227-29466c17df3e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:34.176504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258599d7-115a-4c68-8c13-d9a0ead4e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:34.177113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258599d7-115a-4c68-8c13-d9a0ead4e254 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:34.177389 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258599d7-115a-4c68-8c13-d9a0ead4e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:34.177686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258599d7-115a-4c68-8c13-d9a0ead4e254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.178461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258599d7-115a-4c68-8c13-d9a0ead4e254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:34.180723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258599d7-115a-4c68-8c13-d9a0ead4e254 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:34.212069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1208/2420] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:34 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:34.216630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f8d5d-0c44-4de6-b27c-dff30e832fc7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:34.228780 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f8d5d-0c44-4de6-b27c-dff30e832fc7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:34.290490 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee79fdf-6570-44f6-9fe2-a29fdf4302b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:34.291126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee79fdf-6570-44f6-9fe2-a29fdf4302b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:34.291449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee79fdf-6570-44f6-9fe2-a29fdf4302b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:34.291711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee79fdf-6570-44f6-9fe2-a29fdf4302b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.292336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee79fdf-6570-44f6-9fe2-a29fdf4302b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:34.294310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee79fdf-6570-44f6-9fe2-a29fdf4302b6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:34.324691 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1213/2421] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:34 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:34.328761 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258599d7-115a-4c68-8c13-d9a0ead4e254 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:34.342045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258599d7-115a-4c68-8c13-d9a0ead4e254 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:34.347735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02175f64-260b-4d4a-b139-5056fda44fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:34.348322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02175f64-260b-4d4a-b139-5056fda44fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:34.348599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02175f64-260b-4d4a-b139-5056fda44fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:34.348866 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02175f64-260b-4d4a-b139-5056fda44fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.349596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02175f64-260b-4d4a-b139-5056fda44fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:34.351692 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02175f64-260b-4d4a-b139-5056fda44fd1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:34.366121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1209/2422] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:34.371297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee79fdf-6570-44f6-9fe2-a29fdf4302b6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:34.382650 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee79fdf-6570-44f6-9fe2-a29fdf4302b6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:34.387664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14fd4d-684e-4ef8-b80c-be71c891d4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:34.388386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14fd4d-684e-4ef8-b80c-be71c891d4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:34.388797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14fd4d-684e-4ef8-b80c-be71c891d4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:34.389081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14fd4d-684e-4ef8-b80c-be71c891d4b6 admin admin] PATH_INFO: `/v3/projects/3325196b1f324c9f870a9550bb5206fd/users/1f43a00fb24c4f4994fb89123d365769/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.406959 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14fd4d-684e-4ef8-b80c-be71c891d4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3325196b1f324c9f870a9550bb5206fd, user_id=1f43a00fb24c4f4994fb89123d365769, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:34.411465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14fd4d-684e-4ef8-b80c-be71c891d4b6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:34.427233 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1214/2423] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:34 2022] PUT /identity/v3/projects/3325196b1f324c9f870a9550bb5206fd/users/1f43a00fb24c4f4994fb89123d365769/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:34.431743 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02175f64-260b-4d4a-b139-5056fda44fd1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:34.442805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02175f64-260b-4d4a-b139-5056fda44fd1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:34.465641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:34.502724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5725c3a-1f52-4dfe-a582-467cbaa60a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:34.503328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5725c3a-1f52-4dfe-a582-467cbaa60a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:34.503609 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5725c3a-1f52-4dfe-a582-467cbaa60a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:34.503883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5725c3a-1f52-4dfe-a582-467cbaa60a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.504529 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5725c3a-1f52-4dfe-a582-467cbaa60a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:34.506145 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5725c3a-1f52-4dfe-a582-467cbaa60a4d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:34.516187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1210/2424] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:34.520177 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14fd4d-684e-4ef8-b80c-be71c891d4b6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:34.526138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:34.533618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14fd4d-684e-4ef8-b80c-be71c891d4b6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:34.540198 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:34.540965 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:34.541244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:34.541514 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] PATH_INFO: `/v3/projects/3325196b1f324c9f870a9550bb5206fd/users/1f43a00fb24c4f4994fb89123d365769/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.543700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3325196b1f324c9f870a9550bb5206fd, user_id=1f43a00fb24c4f4994fb89123d365769, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:34.546606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:34.558551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1215/2425] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:34 2022] PUT /identity/v3/projects/3325196b1f324c9f870a9550bb5206fd/users/1f43a00fb24c4f4994fb89123d365769/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:34.569381 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a83a1c-87b8-4e30-8ec6-13600fe07683 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:34.569563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a83a1c-87b8-4e30-8ec6-13600fe07683 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:34.569794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a83a1c-87b8-4e30-8ec6-13600fe07683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.602029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-78a83a1c-87b8-4e30-8ec6-13600fe07683 None None] MFA Rules not processed for user `1f43a00fb24c4f4994fb89123d365769`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:34.603382 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-78a83a1c-87b8-4e30-8ec6-13600fe07683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:34.675623 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1211/2426] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:34.687852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:34.689809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:34.758423 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daf5df-a7bf-4adc-b970-0335e1132e58 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:34.762245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c81bf-a601-4604-88cd-bb34255dc90a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:34.762849 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c81bf-a601-4604-88cd-bb34255dc90a service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:34.763128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c81bf-a601-4604-88cd-bb34255dc90a service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:34.763400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c81bf-a601-4604-88cd-bb34255dc90a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:34.819364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c81bf-a601-4604-88cd-bb34255dc90a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:34.820713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c81bf-a601-4604-88cd-bb34255dc90a service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:34.829778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1216/2427] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:36.334073 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a83a1c-87b8-4e30-8ec6-13600fe07683 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:36.336023 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-78a83a1c-87b8-4e30-8ec6-13600fe07683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:36.418045 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a83a1c-87b8-4e30-8ec6-13600fe07683 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:36.422430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:36.423051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:36.423325 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:36.423594 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] PATH_INFO: `/v3/users/8f65dbb7cd6343a3aeab4f9ef20fe57e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:36.424697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f65dbb7cd6343a3aeab4f9ef20fe57e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:36.426642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:36.480076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f65dbb7cd6343a3aeab4f9ef20fe57e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:36.480653 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f65dbb7cd6343a3aeab4f9ef20fe57e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:36.481862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:36.496634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f65dbb7cd6343a3aeab4f9ef20fe57e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:36.504088 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f65dbb7cd6343a3aeab4f9ef20fe57e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:36.505061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f65dbb7cd6343a3aeab4f9ef20fe57e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:36.520918 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1212/2428] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:36 2022] DELETE /identity/v3/users/8f65dbb7cd6343a3aeab4f9ef20fe57e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:36.525604 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461f75-8925-48bf-981c-7bf98a273cc8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:36.526217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461f75-8925-48bf-981c-7bf98a273cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:36.526217 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461f75-8925-48bf-981c-7bf98a273cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:36.546201 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-07461f75-8925-48bf-981c-7bf98a273cc8 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:36.547398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-07461f75-8925-48bf-981c-7bf98a273cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:36.608377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1217/2429] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:38:36.619008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:36.620505 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:36.692222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa46fa-08cc-458b-be56-a6b43a923a4d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:36.696852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10ccb6-d23d-4fb2-b12b-a707c95066a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:36.697671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10ccb6-d23d-4fb2-b12b-a707c95066a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:36.697941 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10ccb6-d23d-4fb2-b12b-a707c95066a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:36.698233 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10ccb6-d23d-4fb2-b12b-a707c95066a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:36.708323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10ccb6-d23d-4fb2-b12b-a707c95066a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:36.709817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10ccb6-d23d-4fb2-b12b-a707c95066a2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:36.719431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1213/2430] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:37.007882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461f75-8925-48bf-981c-7bf98a273cc8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:37.009598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-07461f75-8925-48bf-981c-7bf98a273cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:37.021871 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461f75-8925-48bf-981c-7bf98a273cc8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:37.028056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:37.028793 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.029139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.029479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] PATH_INFO: `/v3/projects/7bb98db6ee62489a9bfb438dcc2feef5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.030710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bb98db6ee62489a9bfb438dcc2feef5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:37.032685 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:37.077988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:37.086471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bb98db6ee62489a9bfb438dcc2feef5', 'request_id': 'req-221ad8ca-7d4b-4d48-9cac-802a456be911'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:37.094561 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bb98db6ee62489a9bfb438dcc2feef5', 'request_id': 'req-221ad8ca-7d4b-4d48-9cac-802a456be911'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:37.101537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bb98db6ee62489a9bfb438dcc2feef5', 'request_id': 'req-221ad8ca-7d4b-4d48-9cac-802a456be911'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:37.111882 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] The token cache is being invalidate because project 7bb98db6ee62489a9bfb438dcc2feef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:37.112174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:37.112818 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-221ad8ca-7d4b-4d48-9cac-802a456be911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:37.114387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1218/2431] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:37 2022] DELETE /identity/v3/projects/7bb98db6ee62489a9bfb438dcc2feef5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:37.120051 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9e0af-8e6f-44cc-b4f5-be2cd67cca5d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.120340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9e0af-8e6f-44cc-b4f5-be2cd67cca5d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.120610 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9e0af-8e6f-44cc-b4f5-be2cd67cca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.140830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-45b9e0af-8e6f-44cc-b4f5-be2cd67cca5d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:37.142016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-45b9e0af-8e6f-44cc-b4f5-be2cd67cca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:37.206713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1214/2432] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:38:37.275835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361f64-b97d-4a89-a66c-b58151afcb2d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.276107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361f64-b97d-4a89-a66c-b58151afcb2d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.276303 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361f64-b97d-4a89-a66c-b58151afcb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.300187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0e361f64-b97d-4a89-a66c-b58151afcb2d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:37.301377 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0e361f64-b97d-4a89-a66c-b58151afcb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:37.310350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1219/2433] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:37.316819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9e0af-8e6f-44cc-b4f5-be2cd67cca5d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:37.327942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9e0af-8e6f-44cc-b4f5-be2cd67cca5d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:37.333785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:37.334420 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.335238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.335238 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.335807 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:38:37.336105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:37.337682 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:37.350154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1215/2434] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:38:37.354589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e361f64-b97d-4a89-a66c-b58151afcb2d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:37.364302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e361f64-b97d-4a89-a66c-b58151afcb2d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:37.370458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af763da4-f906-4210-836e-56db0cc02419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:37.371049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763da4-f906-4210-836e-56db0cc02419 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.371328 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763da4-f906-4210-836e-56db0cc02419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.371593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763da4-f906-4210-836e-56db0cc02419 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.372222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af763da4-f906-4210-836e-56db0cc02419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:37.374121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af763da4-f906-4210-836e-56db0cc02419 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:37.404690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1220/2435] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:37 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:37.409870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:37.420686 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933f6d-7222-40c9-8908-6c64e2339c80 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:37.472209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:37.483424 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750162-8f70-46b7-9ccf-0bfac3065bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:37.484351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750162-8f70-46b7-9ccf-0bfac3065bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.485220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750162-8f70-46b7-9ccf-0bfac3065bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.485611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750162-8f70-46b7-9ccf-0bfac3065bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.486904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed750162-8f70-46b7-9ccf-0bfac3065bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:37.490259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed750162-8f70-46b7-9ccf-0bfac3065bf9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:37.517642 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1216/2436] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:37 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:37.523700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af763da4-f906-4210-836e-56db0cc02419 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:37.534481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af763da4-f906-4210-836e-56db0cc02419 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:37.539931 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e8216-ba9d-4296-a045-956144d3f7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:37.540547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e8216-ba9d-4296-a045-956144d3f7dd admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.540913 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e8216-ba9d-4296-a045-956144d3f7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.541186 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e8216-ba9d-4296-a045-956144d3f7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.541816 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e8216-ba9d-4296-a045-956144d3f7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:37.543447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e8216-ba9d-4296-a045-956144d3f7dd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:37.554910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1221/2437] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:37.558839 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750162-8f70-46b7-9ccf-0bfac3065bf9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:37.569247 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750162-8f70-46b7-9ccf-0bfac3065bf9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:37.574030 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a696a3-9948-45f1-bb85-bb07a2e05f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:37.574647 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a696a3-9948-45f1-bb85-bb07a2e05f03 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.574922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a696a3-9948-45f1-bb85-bb07a2e05f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.575306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a696a3-9948-45f1-bb85-bb07a2e05f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.575835 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a696a3-9948-45f1-bb85-bb07a2e05f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:37.577409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a696a3-9948-45f1-bb85-bb07a2e05f03 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:37.588584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1217/2438] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:37.592912 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e8216-ba9d-4296-a045-956144d3f7dd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:37.603745 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e8216-ba9d-4296-a045-956144d3f7dd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:37.608371 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:37.609012 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.609283 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.609551 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] PATH_INFO: `/v3/projects/c5e944deacad4e218281351e87e253c7/users/5412799cb6774f54a2fda25bcbc4e4ee/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.624034 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e944deacad4e218281351e87e253c7, user_id=5412799cb6774f54a2fda25bcbc4e4ee, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:37.626681 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:37.640305 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1222/2439] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:37 2022] PUT /identity/v3/projects/c5e944deacad4e218281351e87e253c7/users/5412799cb6774f54a2fda25bcbc4e4ee/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:37.650803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47577c-5662-4ee6-a546-3fcbc507db54 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.651112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47577c-5662-4ee6-a546-3fcbc507db54 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.651380 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47577c-5662-4ee6-a546-3fcbc507db54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.681066 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ca47577c-5662-4ee6-a546-3fcbc507db54 None None] MFA Rules not processed for user `5412799cb6774f54a2fda25bcbc4e4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:37.682338 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ca47577c-5662-4ee6-a546-3fcbc507db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:37.749838 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1218/2440] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:37.762687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:37.764682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:37.827042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd20d6-0804-4124-adcf-0e034825b199 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:37.831175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315b24d-f148-4d8f-bfa8-b8f053e377e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:37.831769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b24d-f148-4d8f-bfa8-b8f053e377e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:37.832040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b24d-f148-4d8f-bfa8-b8f053e377e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:37.832304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b24d-f148-4d8f-bfa8-b8f053e377e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:37.841350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315b24d-f148-4d8f-bfa8-b8f053e377e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:37.842809 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315b24d-f148-4d8f-bfa8-b8f053e377e4 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:37.851939 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1223/2441] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:41.588069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47577c-5662-4ee6-a546-3fcbc507db54 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:41.589513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ca47577c-5662-4ee6-a546-3fcbc507db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:41.602586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47577c-5662-4ee6-a546-3fcbc507db54 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:41.607599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcff4a7-fc1d-42dc-a5b0-5a38b715dc4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:41.608213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcff4a7-fc1d-42dc-a5b0-5a38b715dc4f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:41.608491 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcff4a7-fc1d-42dc-a5b0-5a38b715dc4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:41.608767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcff4a7-fc1d-42dc-a5b0-5a38b715dc4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:41.610773 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cdcff4a7-fc1d-42dc-a5b0-5a38b715dc4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:41.682297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcff4a7-fc1d-42dc-a5b0-5a38b715dc4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:41.683751 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcff4a7-fc1d-42dc-a5b0-5a38b715dc4f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:41.693555 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1219/2442] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:47.848006 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:51.689173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:52.735827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315b24d-f148-4d8f-bfa8-b8f053e377e4 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:52.815690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315b24d-f148-4d8f-bfa8-b8f053e377e4 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:52.820790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22e865-5478-474b-861d-696768478819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:52.821424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22e865-5478-474b-861d-696768478819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:52.821760 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22e865-5478-474b-861d-696768478819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:52.822005 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22e865-5478-474b-861d-696768478819 admin admin] PATH_INFO: `/v3/users/5412799cb6774f54a2fda25bcbc4e4ee` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:52.823664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22e865-5478-474b-861d-696768478819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5412799cb6774f54a2fda25bcbc4e4ee)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:52.825834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22e865-5478-474b-861d-696768478819 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:52.877755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c22e865-5478-474b-861d-696768478819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5412799cb6774f54a2fda25bcbc4e4ee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:52.878417 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c22e865-5478-474b-861d-696768478819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5412799cb6774f54a2fda25bcbc4e4ee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:52.892020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c22e865-5478-474b-861d-696768478819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5412799cb6774f54a2fda25bcbc4e4ee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:52.905475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c22e865-5478-474b-861d-696768478819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5412799cb6774f54a2fda25bcbc4e4ee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:52.906227 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2c22e865-5478-474b-861d-696768478819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5412799cb6774f54a2fda25bcbc4e4ee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:52.927482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1224/2443] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:38:52 2022] DELETE /identity/v3/users/5412799cb6774f54a2fda25bcbc4e4ee => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:52.932054 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f23a3-0689-4cef-b483-ea3562c827f3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:52.933600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f23a3-0689-4cef-b483-ea3562c827f3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:52.934044 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f23a3-0689-4cef-b483-ea3562c827f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:52.954173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-113f23a3-0689-4cef-b483-ea3562c827f3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:52.955304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-113f23a3-0689-4cef-b483-ea3562c827f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:53.015934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1220/2444] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:53.027547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22e865-5478-474b-861d-696768478819 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:53.029055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c22e865-5478-474b-861d-696768478819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:53.095427 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22e865-5478-474b-861d-696768478819 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:53.100205 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5aff13-0edc-4e4c-bf45-155e59a17c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:53.100823 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aff13-0edc-4e4c-bf45-155e59a17c62 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.101095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aff13-0edc-4e4c-bf45-155e59a17c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.101365 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aff13-0edc-4e4c-bf45-155e59a17c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.111139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aff13-0edc-4e4c-bf45-155e59a17c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:53.112559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aff13-0edc-4e4c-bf45-155e59a17c62 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:53.122478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1225/2445] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:38:53.402767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f23a3-0689-4cef-b483-ea3562c827f3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:53.404340 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-113f23a3-0689-4cef-b483-ea3562c827f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:53.416930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f23a3-0689-4cef-b483-ea3562c827f3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:53.424105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:53.424601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.424876 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.425148 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] PATH_INFO: `/v3/projects/c5e944deacad4e218281351e87e253c7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.426441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e944deacad4e218281351e87e253c7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:53.429565 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:53.479212 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e944deacad4e218281351e87e253c7', 'request_id': 'req-21b6376c-2eb8-4158-8535-99ba78afcc38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:53.485215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e944deacad4e218281351e87e253c7', 'request_id': 'req-21b6376c-2eb8-4158-8535-99ba78afcc38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:53.495226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e944deacad4e218281351e87e253c7', 'request_id': 'req-21b6376c-2eb8-4158-8535-99ba78afcc38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:53.505634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] The token cache is being invalidate because project c5e944deacad4e218281351e87e253c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:38:53.505921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:53.506586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-21b6376c-2eb8-4158-8535-99ba78afcc38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:38:53.508193 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1221/2446] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:38:53 2022] DELETE /identity/v3/projects/c5e944deacad4e218281351e87e253c7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:53.513956 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474ca7-5893-4d79-aa74-86dec14b747a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.514289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474ca7-5893-4d79-aa74-86dec14b747a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.514534 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474ca7-5893-4d79-aa74-86dec14b747a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.535037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-90474ca7-5893-4d79-aa74-86dec14b747a None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:53.536232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-90474ca7-5893-4d79-aa74-86dec14b747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:53.554072 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:53.599998 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1226/2447] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:53.608046 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce416b4-056b-4831-8b4a-c7e15387a89d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.608360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce416b4-056b-4831-8b4a-c7e15387a89d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.608644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce416b4-056b-4831-8b4a-c7e15387a89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.628987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5ce416b4-056b-4831-8b4a-c7e15387a89d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:53.630226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ce416b4-056b-4831-8b4a-c7e15387a89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:53.638608 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1222/2448] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:53.642925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90474ca7-5893-4d79-aa74-86dec14b747a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:53.652736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90474ca7-5893-4d79-aa74-86dec14b747a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:53.657626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:53.658225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.658499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.658770 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.659436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:38:53.659737 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:53.661317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:53.673858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1227/2449] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:38:53.678580 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce416b4-056b-4831-8b4a-c7e15387a89d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:53.689916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce416b4-056b-4831-8b4a-c7e15387a89d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:53.694676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf54d74-dce2-43c7-9fbe-26428fff2356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:53.695261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf54d74-dce2-43c7-9fbe-26428fff2356 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.695538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf54d74-dce2-43c7-9fbe-26428fff2356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.695812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf54d74-dce2-43c7-9fbe-26428fff2356 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.696435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf54d74-dce2-43c7-9fbe-26428fff2356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:53.698863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf54d74-dce2-43c7-9fbe-26428fff2356 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:53.729374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1223/2450] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:53 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:53.734522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:53.745981 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf93e0-f600-4453-a115-7e6cb35f8ce8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:53.809289 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50990e2e-1ddb-4d22-8876-42dba1498ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:53.809899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50990e2e-1ddb-4d22-8876-42dba1498ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.810226 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50990e2e-1ddb-4d22-8876-42dba1498ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.810513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50990e2e-1ddb-4d22-8876-42dba1498ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.811158 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50990e2e-1ddb-4d22-8876-42dba1498ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:53.813163 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50990e2e-1ddb-4d22-8876-42dba1498ae1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:53.838362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1228/2451] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:53 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:53.840903 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf54d74-dce2-43c7-9fbe-26428fff2356 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:53.852306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf54d74-dce2-43c7-9fbe-26428fff2356 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:53.858028 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428170d-ab1f-41dc-9f94-a4a428a69d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:53.858654 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428170d-ab1f-41dc-9f94-a4a428a69d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.858949 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428170d-ab1f-41dc-9f94-a4a428a69d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.859199 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428170d-ab1f-41dc-9f94-a4a428a69d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.859836 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428170d-ab1f-41dc-9f94-a4a428a69d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:53.861440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428170d-ab1f-41dc-9f94-a4a428a69d99 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:53.871934 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1224/2452] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:53.876096 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50990e2e-1ddb-4d22-8876-42dba1498ae1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:53.887613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50990e2e-1ddb-4d22-8876-42dba1498ae1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:53.892858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a403b-c79c-4195-bfc8-9973b9d376bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:53.893443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a403b-c79c-4195-bfc8-9973b9d376bb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.893791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a403b-c79c-4195-bfc8-9973b9d376bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.894065 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a403b-c79c-4195-bfc8-9973b9d376bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.894726 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a403b-c79c-4195-bfc8-9973b9d376bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:53.896327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a403b-c79c-4195-bfc8-9973b9d376bb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:53.907535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1229/2453] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:53.911750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428170d-ab1f-41dc-9f94-a4a428a69d99 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:53.922787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428170d-ab1f-41dc-9f94-a4a428a69d99 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:53.927611 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:53.928257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.928536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.928805 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] PATH_INFO: `/v3/projects/59ab1e9b63ae4a7e851eb1de51d64aef/users/ccdebe0f1a1646acbadf5cca7e97c7d2/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.944069 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ab1e9b63ae4a7e851eb1de51d64aef, user_id=ccdebe0f1a1646acbadf5cca7e97c7d2, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:53.947058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:53.960632 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1225/2454] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:53 2022] PUT /identity/v3/projects/59ab1e9b63ae4a7e851eb1de51d64aef/users/ccdebe0f1a1646acbadf5cca7e97c7d2/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:53.966395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b6cf-f888-43ee-831f-9bd3fdd2d833 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:53.966741 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b6cf-f888-43ee-831f-9bd3fdd2d833 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:53.967141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b6cf-f888-43ee-831f-9bd3fdd2d833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:53.991391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2442b6cf-f888-43ee-831f-9bd3fdd2d833 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:53.992504 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2442b6cf-f888-43ee-831f-9bd3fdd2d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:54.068349 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1230/2455] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:38:54.080108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:54.081822 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:54.104450 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:54.141065 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a165af-aeca-4286-b8ba-0bb7842f579e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:54.145015 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeac37-b261-434c-8143-a771db17ce6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:54.145650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeac37-b261-434c-8143-a771db17ce6c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:54.145950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeac37-b261-434c-8143-a771db17ce6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:54.146244 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeac37-b261-434c-8143-a771db17ce6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:54.155754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeac37-b261-434c-8143-a771db17ce6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:54.157208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeac37-b261-434c-8143-a771db17ce6c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:54.166946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1226/2456] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:38:58.393735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0009e-c301-4728-84d6-a7784c4f06a7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:58.394820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0009e-c301-4728-84d6-a7784c4f06a7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:58.395121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0009e-c301-4728-84d6-a7784c4f06a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:58.426787 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2eb0009e-c301-4728-84d6-a7784c4f06a7 None None] MFA Rules not processed for user `ccdebe0f1a1646acbadf5cca7e97c7d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:58.427986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2eb0009e-c301-4728-84d6-a7784c4f06a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:58.447604 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:58.516166 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1231/2457] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:38:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:58.520852 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeac37-b261-434c-8143-a771db17ce6c service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:58.531861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeac37-b261-434c-8143-a771db17ce6c service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:58.537113 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27935a1-f8fc-441d-a95a-8184bca10edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:58.537745 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27935a1-f8fc-441d-a95a-8184bca10edf admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:58.538155 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27935a1-f8fc-441d-a95a-8184bca10edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:58.538467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27935a1-f8fc-441d-a95a-8184bca10edf admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:58.539195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27935a1-f8fc-441d-a95a-8184bca10edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:58.541316 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27935a1-f8fc-441d-a95a-8184bca10edf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:58.572360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1227/2458] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:38:58 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:38:58.576773 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0009e-c301-4728-84d6-a7784c4f06a7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:58.588239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0009e-c301-4728-84d6-a7784c4f06a7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:58.652490 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e5ace-b25f-4fd7-bed8-bb8956e6e85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:58.652977 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e5ace-b25f-4fd7-bed8-bb8956e6e85b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:58.653263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e5ace-b25f-4fd7-bed8-bb8956e6e85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:58.653536 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e5ace-b25f-4fd7-bed8-bb8956e6e85b admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:58.654189 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e5ace-b25f-4fd7-bed8-bb8956e6e85b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:58.656138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e5ace-b25f-4fd7-bed8-bb8956e6e85b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:58.679029 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1232/2459] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:38:58 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:38:58.683225 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27935a1-f8fc-441d-a95a-8184bca10edf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:58.694110 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27935a1-f8fc-441d-a95a-8184bca10edf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:58.699327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a89eda-1791-4b9a-98e4-70514575b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:58.700037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a89eda-1791-4b9a-98e4-70514575b32d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:58.700468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a89eda-1791-4b9a-98e4-70514575b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:58.700776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a89eda-1791-4b9a-98e4-70514575b32d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:58.701728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a89eda-1791-4b9a-98e4-70514575b32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:58.703224 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a89eda-1791-4b9a-98e4-70514575b32d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:58.717915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1228/2460] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:38:58.722452 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e5ace-b25f-4fd7-bed8-bb8956e6e85b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:58.733516 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e5ace-b25f-4fd7-bed8-bb8956e6e85b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:58.739352 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2fe81-efc1-4d18-8d6f-b1647dd21c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:58.739352 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2fe81-efc1-4d18-8d6f-b1647dd21c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:58.739794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2fe81-efc1-4d18-8d6f-b1647dd21c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:58.740167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2fe81-efc1-4d18-8d6f-b1647dd21c4b admin admin] PATH_INFO: `/v3/projects/6023341c866c41f48f9210997de52a39/users/02a570a4c3a34b22ab4d9a4be1bdc1fc/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:58.756427 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2fe81-efc1-4d18-8d6f-b1647dd21c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6023341c866c41f48f9210997de52a39, user_id=02a570a4c3a34b22ab4d9a4be1bdc1fc, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:58.759271 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2fe81-efc1-4d18-8d6f-b1647dd21c4b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:58.774145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1233/2461] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:58 2022] PUT /identity/v3/projects/6023341c866c41f48f9210997de52a39/users/02a570a4c3a34b22ab4d9a4be1bdc1fc/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:38:58.778714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a89eda-1791-4b9a-98e4-70514575b32d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:58.789299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a89eda-1791-4b9a-98e4-70514575b32d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:58.845638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fd9a2-5124-46eb-9799-46dfac94a1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:58.846277 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fd9a2-5124-46eb-9799-46dfac94a1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:58.846872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fd9a2-5124-46eb-9799-46dfac94a1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:58.847141 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fd9a2-5124-46eb-9799-46dfac94a1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:58.847775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fd9a2-5124-46eb-9799-46dfac94a1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:58.849373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fd9a2-5124-46eb-9799-46dfac94a1b9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:58.863557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1229/2462] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:38:58.864824 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2fe81-efc1-4d18-8d6f-b1647dd21c4b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:58.875863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2fe81-efc1-4d18-8d6f-b1647dd21c4b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:58.881008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:58.881693 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:58.881976 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:58.882386 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] PATH_INFO: `/v3/projects/6023341c866c41f48f9210997de52a39/users/02a570a4c3a34b22ab4d9a4be1bdc1fc/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:58.884459 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6023341c866c41f48f9210997de52a39, user_id=02a570a4c3a34b22ab4d9a4be1bdc1fc, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:58.887235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:58.898790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1234/2463] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:38:58 2022] PUT /identity/v3/projects/6023341c866c41f48f9210997de52a39/users/02a570a4c3a34b22ab4d9a4be1bdc1fc/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:03.673043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680f25-87a1-4f78-b84c-cf4728723693 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:03.673471 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680f25-87a1-4f78-b84c-cf4728723693 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:03.673616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680f25-87a1-4f78-b84c-cf4728723693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:03.708583 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3f680f25-87a1-4f78-b84c-cf4728723693 None None] MFA Rules not processed for user `02a570a4c3a34b22ab4d9a4be1bdc1fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:03.710710 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3f680f25-87a1-4f78-b84c-cf4728723693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:03.787386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1230/2464] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:03.807073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:03.811546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:03.890484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57572b6-4f70-431f-bbab-5fbb9ab837b2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:03.896216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:03.896838 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:03.897117 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:03.897390 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:03.903875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:03.960533 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:03.962149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:03.972218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1235/2465] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:05.025211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11245152-1df2-4c27-ba5d-b736c26f1f61 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:05.025787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11245152-1df2-4c27-ba5d-b736c26f1f61 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:05.025787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11245152-1df2-4c27-ba5d-b736c26f1f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:05.050544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-11245152-1df2-4c27-ba5d-b736c26f1f61 None None] MFA Rules not processed for user `ccdebe0f1a1646acbadf5cca7e97c7d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:05.051765 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-11245152-1df2-4c27-ba5d-b736c26f1f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:05.059075 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1231/2466] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:05.984937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:05.986479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:06.049054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f9904-49f6-4492-ad9e-5b6dd08e53d4 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:06.053204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:06.053892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:06.054337 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:06.054783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:06.056557 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:06.067024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:06.068446 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:06.077047 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1236/2467] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:15.045354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:16.072843 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=50 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:26.622972 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11245152-1df2-4c27-ba5d-b736c26f1f61 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:26.639542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11245152-1df2-4c27-ba5d-b736c26f1f61 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:26.645454 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ee3ea-2803-4d30-80a5-663c43815d32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:26.646067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ee3ea-2803-4d30-80a5-663c43815d32 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:26.646371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ee3ea-2803-4d30-80a5-663c43815d32 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:26.646637 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ee3ea-2803-4d30-80a5-663c43815d32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:26.648755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-998ee3ea-2803-4d30-80a5-663c43815d32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:26.664237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ee3ea-2803-4d30-80a5-663c43815d32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:26.665798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ee3ea-2803-4d30-80a5-663c43815d32 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:26.678942 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1232/2468] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:26.711970 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:26.713398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:26.780049 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeccca-2a97-4309-a93a-491aee75f663 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:26.785107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:26.785751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:26.786031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:26.786404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] PATH_INFO: `/v3/users/74e077ae6b974b6ab029b53db85e9da7` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:26.787559 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e077ae6b974b6ab029b53db85e9da7)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:26.789488 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:26.839114 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e077ae6b974b6ab029b53db85e9da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:26.839757 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e077ae6b974b6ab029b53db85e9da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:26.852159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e077ae6b974b6ab029b53db85e9da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:26.859174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e077ae6b974b6ab029b53db85e9da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:26.859867 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e077ae6b974b6ab029b53db85e9da7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:26.874159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1237/2469] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:39:26 2022] DELETE /identity/v3/users/74e077ae6b974b6ab029b53db85e9da7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:26.879071 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07935c6f-4608-49cc-82b7-2044237a8dac None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:26.879358 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07935c6f-4608-49cc-82b7-2044237a8dac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:26.879651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07935c6f-4608-49cc-82b7-2044237a8dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:26.899154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-07935c6f-4608-49cc-82b7-2044237a8dac None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:26.900374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-07935c6f-4608-49cc-82b7-2044237a8dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:26.963919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1233/2470] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:26.974565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:26.976107 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:27.037613 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13356e7-4001-4444-97c9-6edca2cfdf22 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:27.042236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28309c92-6a4a-4916-998a-1bccb7e853bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:27.042844 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28309c92-6a4a-4916-998a-1bccb7e853bb service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:27.043167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28309c92-6a4a-4916-998a-1bccb7e853bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:27.043387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28309c92-6a4a-4916-998a-1bccb7e853bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:27.053440 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28309c92-6a4a-4916-998a-1bccb7e853bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:27.054898 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28309c92-6a4a-4916-998a-1bccb7e853bb service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:27.064387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1238/2471] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:27.356271 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07935c6f-4608-49cc-82b7-2044237a8dac None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:27.357778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-07935c6f-4608-49cc-82b7-2044237a8dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:27.371981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07935c6f-4608-49cc-82b7-2044237a8dac None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:27.378118 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:27.379029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:27.379368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:27.379775 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] PATH_INFO: `/v3/projects/df4a8e78415b43bd9c4a1201e15c9155` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:27.381089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4a8e78415b43bd9c4a1201e15c9155)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:27.384235 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:27.432122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4a8e78415b43bd9c4a1201e15c9155', 'request_id': 'req-5fe28b11-2135-480f-9fb0-329b3cfa560e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.438303 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4a8e78415b43bd9c4a1201e15c9155', 'request_id': 'req-5fe28b11-2135-480f-9fb0-329b3cfa560e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.448150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4a8e78415b43bd9c4a1201e15c9155', 'request_id': 'req-5fe28b11-2135-480f-9fb0-329b3cfa560e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.459322 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] The token cache is being invalidate because project df4a8e78415b43bd9c4a1201e15c9155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:39:27.459635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.460269 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.461770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1234/2472] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:39:27 2022] DELETE /identity/v3/projects/df4a8e78415b43bd9c4a1201e15c9155 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:27.465954 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28309c92-6a4a-4916-998a-1bccb7e853bb service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:27.476056 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28309c92-6a4a-4916-998a-1bccb7e853bb service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:27.494972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:27.555654 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:27.556204 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:27.556481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:27.556752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] PATH_INFO: `/v3/users/1f43a00fb24c4f4994fb89123d365769` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:27.557876 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f43a00fb24c4f4994fb89123d365769)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:27.560240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:27.613990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f43a00fb24c4f4994fb89123d365769'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.614667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f43a00fb24c4f4994fb89123d365769'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.626995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f43a00fb24c4f4994fb89123d365769'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.634175 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f43a00fb24c4f4994fb89123d365769'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.634966 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ae47638d-a074-403d-b27c-e0b0bee16719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f43a00fb24c4f4994fb89123d365769'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:27.653995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1239/2473] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:39:27 2022] DELETE /identity/v3/users/1f43a00fb24c4f4994fb89123d365769 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:27.981628 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:27.983283 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:27.998687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28b11-2135-480f-9fb0-329b3cfa560e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.057925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.058622 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.058897 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.059164 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] PATH_INFO: `/v3/projects/3325196b1f324c9f870a9550bb5206fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.060320 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3325196b1f324c9f870a9550bb5206fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.062286 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.074584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:28.115472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3325196b1f324c9f870a9550bb5206fd', 'request_id': 'req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:28.121596 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3325196b1f324c9f870a9550bb5206fd', 'request_id': 'req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:28.132544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3325196b1f324c9f870a9550bb5206fd', 'request_id': 'req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:28.145534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] The token cache is being invalidate because project 3325196b1f324c9f870a9550bb5206fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:39:28.145819 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:28.146474 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4edb2aaa-a6f9-4d2b-83eb-b644b79705fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:28.147996 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1235/2474] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:39:27 2022] DELETE /identity/v3/projects/3325196b1f324c9f870a9550bb5206fd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:28.153111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1936493-2186-45c4-a05e-6448f3b74206 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.153655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1936493-2186-45c4-a05e-6448f3b74206 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.153655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1936493-2186-45c4-a05e-6448f3b74206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.173364 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e1936493-2186-45c4-a05e-6448f3b74206 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:28.174662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e1936493-2186-45c4-a05e-6448f3b74206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:28.234858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1240/2475] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:28.242905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113115ac-7032-4d8b-a265-035fbc9e2472 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.243188 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113115ac-7032-4d8b-a265-035fbc9e2472 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.243462 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113115ac-7032-4d8b-a265-035fbc9e2472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.264599 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-113115ac-7032-4d8b-a265-035fbc9e2472 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:28.265886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-113115ac-7032-4d8b-a265-035fbc9e2472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:28.275211 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1236/2476] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:28.279735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1936493-2186-45c4-a05e-6448f3b74206 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.289705 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1936493-2186-45c4-a05e-6448f3b74206 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.294615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.295215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.295496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.295771 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.296419 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:39:28.296724 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.298427 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.310167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1241/2477] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:39:28.315061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113115ac-7032-4d8b-a265-035fbc9e2472 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.327944 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113115ac-7032-4d8b-a265-035fbc9e2472 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.333552 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49913694-9f14-4fc9-ba22-80248394687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.334142 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49913694-9f14-4fc9-ba22-80248394687c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.334451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49913694-9f14-4fc9-ba22-80248394687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.334728 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49913694-9f14-4fc9-ba22-80248394687c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.335360 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49913694-9f14-4fc9-ba22-80248394687c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.337259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49913694-9f14-4fc9-ba22-80248394687c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.368085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1237/2478] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:39:28 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:28.372472 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.382669 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea88fe4-820f-4d42-9fcd-3a9243846a82 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.409755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:28.443846 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443ccf-05ec-42af-969a-fbfdc8fa6313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.444444 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443ccf-05ec-42af-969a-fbfdc8fa6313 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.444721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443ccf-05ec-42af-969a-fbfdc8fa6313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.444985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443ccf-05ec-42af-969a-fbfdc8fa6313 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.445617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443ccf-05ec-42af-969a-fbfdc8fa6313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.447550 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443ccf-05ec-42af-969a-fbfdc8fa6313 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.478236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1242/2479] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:39:28 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:39:28.482446 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49913694-9f14-4fc9-ba22-80248394687c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.493701 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49913694-9f14-4fc9-ba22-80248394687c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.498945 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4970c-bd84-4cae-8707-13abcc35377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.499531 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4970c-bd84-4cae-8707-13abcc35377b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.499808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4970c-bd84-4cae-8707-13abcc35377b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.500077 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4970c-bd84-4cae-8707-13abcc35377b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.500714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4970c-bd84-4cae-8707-13abcc35377b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.502451 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4970c-bd84-4cae-8707-13abcc35377b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.512754 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1238/2480] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:28.516856 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443ccf-05ec-42af-969a-fbfdc8fa6313 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.527055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443ccf-05ec-42af-969a-fbfdc8fa6313 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.532320 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4fef6-b43c-49d5-a93b-980e945f04e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.532912 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fef6-b43c-49d5-a93b-980e945f04e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.533193 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fef6-b43c-49d5-a93b-980e945f04e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.533470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fef6-b43c-49d5-a93b-980e945f04e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.534137 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4fef6-b43c-49d5-a93b-980e945f04e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.535733 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4fef6-b43c-49d5-a93b-980e945f04e3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.546523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1243/2481] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:28.550789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4970c-bd84-4cae-8707-13abcc35377b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.562386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4970c-bd84-4cae-8707-13abcc35377b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.567861 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a306a8c-d2b6-4176-ad17-9878295d95c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.568512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a306a8c-d2b6-4176-ad17-9878295d95c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.568787 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a306a8c-d2b6-4176-ad17-9878295d95c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.569058 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a306a8c-d2b6-4176-ad17-9878295d95c6 admin admin] PATH_INFO: `/v3/projects/94039ca1ad9a49bcae0e560a242c42f2/users/71d5401b7e8a46a3a8786f00a5102ef8/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.584169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a306a8c-d2b6-4176-ad17-9878295d95c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94039ca1ad9a49bcae0e560a242c42f2, user_id=71d5401b7e8a46a3a8786f00a5102ef8, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.586813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a306a8c-d2b6-4176-ad17-9878295d95c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.601253 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1239/2482] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:39:28 2022] PUT /identity/v3/projects/94039ca1ad9a49bcae0e560a242c42f2/users/71d5401b7e8a46a3a8786f00a5102ef8/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:28.611879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c08ec4-4e12-4deb-bc77-79273e1fcfea None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.612160 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c08ec4-4e12-4deb-bc77-79273e1fcfea None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.612443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c08ec4-4e12-4deb-bc77-79273e1fcfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.641324 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-85c08ec4-4e12-4deb-bc77-79273e1fcfea None None] MFA Rules not processed for user `71d5401b7e8a46a3a8786f00a5102ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:28.642692 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-85c08ec4-4e12-4deb-bc77-79273e1fcfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:28.709282 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1244/2483] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:28.714569 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a306a8c-d2b6-4176-ad17-9878295d95c6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.778914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a306a8c-d2b6-4176-ad17-9878295d95c6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.784726 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb61347-8599-4bfe-ac26-dac548ba7b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.785341 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61347-8599-4bfe-ac26-dac548ba7b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.785619 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61347-8599-4bfe-ac26-dac548ba7b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.785889 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61347-8599-4bfe-ac26-dac548ba7b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.786566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb61347-8599-4bfe-ac26-dac548ba7b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.788486 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb61347-8599-4bfe-ac26-dac548ba7b48 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.817559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1240/2484] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:39:28 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:28.821603 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c08ec4-4e12-4deb-bc77-79273e1fcfea None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.833188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c08ec4-4e12-4deb-bc77-79273e1fcfea None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.895239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fac85-59c2-476a-a5c4-9f8a51716083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.895858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fac85-59c2-476a-a5c4-9f8a51716083 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.896134 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fac85-59c2-476a-a5c4-9f8a51716083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.896406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fac85-59c2-476a-a5c4-9f8a51716083 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.897020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fac85-59c2-476a-a5c4-9f8a51716083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.898964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fac85-59c2-476a-a5c4-9f8a51716083 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.923333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1245/2485] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:39:28 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:28.927815 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb61347-8599-4bfe-ac26-dac548ba7b48 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.939346 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb61347-8599-4bfe-ac26-dac548ba7b48 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.944841 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31bcc1-fa4d-42b2-9c78-78d93caac1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.945452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31bcc1-fa4d-42b2-9c78-78d93caac1df admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.945732 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31bcc1-fa4d-42b2-9c78-78d93caac1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.945997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31bcc1-fa4d-42b2-9c78-78d93caac1df admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.946676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31bcc1-fa4d-42b2-9c78-78d93caac1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.948372 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31bcc1-fa4d-42b2-9c78-78d93caac1df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.960643 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1241/2486] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:28.966424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fac85-59c2-476a-a5c4-9f8a51716083 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:28.977668 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fac85-59c2-476a-a5c4-9f8a51716083 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:28.983145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f638fe-f600-4649-b919-baad6e5083c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:28.983794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f638fe-f600-4649-b919-baad6e5083c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:28.984064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f638fe-f600-4649-b919-baad6e5083c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:28.984316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f638fe-f600-4649-b919-baad6e5083c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:28.984961 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f638fe-f600-4649-b919-baad6e5083c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:28.986675 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f638fe-f600-4649-b919-baad6e5083c4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:28.996911 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1246/2487] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:29.001343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31bcc1-fa4d-42b2-9c78-78d93caac1df admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:29.011786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31bcc1-fa4d-42b2-9c78-78d93caac1df admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:29.016857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d086cb-2839-44b7-8054-cef7caec17d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:29.017497 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d086cb-2839-44b7-8054-cef7caec17d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.017776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d086cb-2839-44b7-8054-cef7caec17d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.018048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d086cb-2839-44b7-8054-cef7caec17d9 admin admin] PATH_INFO: `/v3/projects/0523235af50f4f01a6d4d11fd5b61e19/users/71664ecc4360430096e45658ad4dfff4/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.028033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:29.034616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d086cb-2839-44b7-8054-cef7caec17d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0523235af50f4f01a6d4d11fd5b61e19, user_id=71664ecc4360430096e45658ad4dfff4, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:29.038261 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d086cb-2839-44b7-8054-cef7caec17d9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:29.053032 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1242/2488] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:39:29 2022] PUT /identity/v3/projects/0523235af50f4f01a6d4d11fd5b61e19/users/71664ecc4360430096e45658ad4dfff4/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:29.064413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9e1af-55f8-4220-9766-c9b963eb13b3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.064739 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9e1af-55f8-4220-9766-c9b963eb13b3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.065015 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9e1af-55f8-4220-9766-c9b963eb13b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.092615 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ebd9e1af-55f8-4220-9766-c9b963eb13b3 None None] MFA Rules not processed for user `71664ecc4360430096e45658ad4dfff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:29.093821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ebd9e1af-55f8-4220-9766-c9b963eb13b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:29.098188 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:29.162617 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1247/2489] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:29.167764 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d086cb-2839-44b7-8054-cef7caec17d9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:29.234858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d086cb-2839-44b7-8054-cef7caec17d9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:29.240159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3b61e-5164-4035-93b8-779e5a89e7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:29.240755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3b61e-5164-4035-93b8-779e5a89e7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.241022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3b61e-5164-4035-93b8-779e5a89e7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.241290 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3b61e-5164-4035-93b8-779e5a89e7c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.241908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3b61e-5164-4035-93b8-779e5a89e7c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:29.243808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3b61e-5164-4035-93b8-779e5a89e7c5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:29.277427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1243/2490] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:39:29 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:29.281702 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9e1af-55f8-4220-9766-c9b963eb13b3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:29.293081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9e1af-55f8-4220-9766-c9b963eb13b3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:29.352642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109f1f7-5a01-4b80-ab85-34e532e80da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:29.353254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109f1f7-5a01-4b80-ab85-34e532e80da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.353531 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109f1f7-5a01-4b80-ab85-34e532e80da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.353801 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109f1f7-5a01-4b80-ab85-34e532e80da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.354465 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109f1f7-5a01-4b80-ab85-34e532e80da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:29.356373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109f1f7-5a01-4b80-ab85-34e532e80da4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:29.384607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1248/2491] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:39:29 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:29.388799 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3b61e-5164-4035-93b8-779e5a89e7c5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:29.400504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3b61e-5164-4035-93b8-779e5a89e7c5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:29.405725 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb17b3c-8db3-4bef-bbdc-88a1b6866143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:29.406347 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb17b3c-8db3-4bef-bbdc-88a1b6866143 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.406627 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb17b3c-8db3-4bef-bbdc-88a1b6866143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.406905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb17b3c-8db3-4bef-bbdc-88a1b6866143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.408093 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb17b3c-8db3-4bef-bbdc-88a1b6866143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:29.409693 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb17b3c-8db3-4bef-bbdc-88a1b6866143 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:29.420227 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1244/2492] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:29.424815 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109f1f7-5a01-4b80-ab85-34e532e80da4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:29.435155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109f1f7-5a01-4b80-ab85-34e532e80da4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:29.440479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee4ae-3a3c-4f9c-9970-41a4bcb0f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:29.441155 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee4ae-3a3c-4f9c-9970-41a4bcb0f9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.441436 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee4ae-3a3c-4f9c-9970-41a4bcb0f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.441697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee4ae-3a3c-4f9c-9970-41a4bcb0f9fc admin admin] PATH_INFO: `/v3/projects/195680ced40a4136adfd43ade8b55734/users/71c21fd2849a4b408d3104bb16211648/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.458366 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee4ae-3a3c-4f9c-9970-41a4bcb0f9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=195680ced40a4136adfd43ade8b55734, user_id=71c21fd2849a4b408d3104bb16211648, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:29.461073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee4ae-3a3c-4f9c-9970-41a4bcb0f9fc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:29.482795 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1249/2493] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:39:29 2022] PUT /identity/v3/projects/195680ced40a4136adfd43ade8b55734/users/71c21fd2849a4b408d3104bb16211648/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:29.489432 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb17b3c-8db3-4bef-bbdc-88a1b6866143 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:29.507067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb17b3c-8db3-4bef-bbdc-88a1b6866143 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:29.577273 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952d76b-009b-4128-9c6e-a68362ecd58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:29.577887 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952d76b-009b-4128-9c6e-a68362ecd58a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.578197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952d76b-009b-4128-9c6e-a68362ecd58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.578510 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952d76b-009b-4128-9c6e-a68362ecd58a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.579154 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952d76b-009b-4128-9c6e-a68362ecd58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:29.580786 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952d76b-009b-4128-9c6e-a68362ecd58a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:29.591668 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1245/2494] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:29.596919 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee4ae-3a3c-4f9c-9970-41a4bcb0f9fc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:29.608747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee4ae-3a3c-4f9c-9970-41a4bcb0f9fc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:29.614431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:29.615138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.615630 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.615987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] PATH_INFO: `/v3/projects/195680ced40a4136adfd43ade8b55734/users/71c21fd2849a4b408d3104bb16211648/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.618138 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195680ced40a4136adfd43ade8b55734, user_id=71c21fd2849a4b408d3104bb16211648, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:29.620752 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:29.634265 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1250/2495] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:39:29 2022] PUT /identity/v3/projects/195680ced40a4136adfd43ade8b55734/users/71c21fd2849a4b408d3104bb16211648/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:29.646484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59168520-b182-4925-b40c-7dbc0eda8780 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.646921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59168520-b182-4925-b40c-7dbc0eda8780 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.647317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59168520-b182-4925-b40c-7dbc0eda8780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.691578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-59168520-b182-4925-b40c-7dbc0eda8780 None None] MFA Rules not processed for user `71c21fd2849a4b408d3104bb16211648`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:29.692750 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-59168520-b182-4925-b40c-7dbc0eda8780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:29.749406 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:29.782403 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1246/2496] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:29.797471 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:29.799664 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:29.877484 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b11300-39e3-4dcd-b94b-14d519e3d444 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:29.881746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51c29b-4802-4833-9886-9f69cdc1318e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:29.882404 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c29b-4802-4833-9886-9f69cdc1318e service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:29.882682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c29b-4802-4833-9886-9f69cdc1318e service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:29.882959 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c29b-4802-4833-9886-9f69cdc1318e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:29.899470 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51c29b-4802-4833-9886-9f69cdc1318e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:29.901074 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51c29b-4802-4833-9886-9f69cdc1318e service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:29.906440 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:29.912543 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1251/2497] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:30.555506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59168520-b182-4925-b40c-7dbc0eda8780 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:30.565937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59168520-b182-4925-b40c-7dbc0eda8780 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:30.570359 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:30.570976 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:30.571250 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:30.571515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:30.633998 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:30.635507 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:30.644762 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1247/2498] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:40.640575 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=36 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:41.831952 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb3213-abb0-4206-af31-e395af2fce40 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:41.832525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb3213-abb0-4206-af31-e395af2fce40 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:41.832525 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb3213-abb0-4206-af31-e395af2fce40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:41.853567 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-aedb3213-abb0-4206-af31-e395af2fce40 None None] MFA Rules not processed for user `2ce997cd838648ddaff729500e87f9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:41.854791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aedb3213-abb0-4206-af31-e395af2fce40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:41.918722 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1252/2499] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:42.168206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:42.172120 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:42.215625 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb3213-abb0-4206-af31-e395af2fce40 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:42.226597 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb3213-abb0-4206-af31-e395af2fce40 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:42.230115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7487-59a2-4c65-a026-500cd7f16c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:42.230749 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7487-59a2-4c65-a026-500cd7f16c47 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:42.231024 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7487-59a2-4c65-a026-500cd7f16c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:42.231296 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e7487-59a2-4c65-a026-500cd7f16c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:42.234524 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ecb6d-c73e-42db-921c-ec431b6a7dc9 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:42.239033 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:42.239832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:42.240222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:42.240500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:42.242607 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:42.253072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:42.254574 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:42.264197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1248/2500] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:42.291231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e7487-59a2-4c65-a026-500cd7f16c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:42.292667 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e7487-59a2-4c65-a026-500cd7f16c47 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:42.302506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1253/2501] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:39:42.645534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:42.648867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:42.762024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac4ab4-3e49-41e5-a8c5-d6204075d940 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:42.767328 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592200d-2b3d-46db-b0a4-ad95b7669689 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:42.767939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592200d-2b3d-46db-b0a4-ad95b7669689 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:42.768203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592200d-2b3d-46db-b0a4-ad95b7669689 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:42.768472 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592200d-2b3d-46db-b0a4-ad95b7669689 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:42.770354 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f592200d-2b3d-46db-b0a4-ad95b7669689 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:42.783296 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592200d-2b3d-46db-b0a4-ad95b7669689 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:42.784871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592200d-2b3d-46db-b0a4-ad95b7669689 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:42.798631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1249/2502] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:39:44.771040 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7487-59a2-4c65-a026-500cd7f16c47 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:44.843443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e7487-59a2-4c65-a026-500cd7f16c47 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:44.848598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:44.849159 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:44.849429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:44.849879 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] PATH_INFO: `/v3/users/71d5401b7e8a46a3a8786f00a5102ef8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:44.851095 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71d5401b7e8a46a3a8786f00a5102ef8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:44.852985 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:44.912313 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:44.914556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71d5401b7e8a46a3a8786f00a5102ef8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:44.915345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71d5401b7e8a46a3a8786f00a5102ef8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:44.929993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71d5401b7e8a46a3a8786f00a5102ef8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:44.937841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71d5401b7e8a46a3a8786f00a5102ef8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:44.938735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71d5401b7e8a46a3a8786f00a5102ef8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:44.957563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1254/2503] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:39:44 2022] DELETE /identity/v3/users/71d5401b7e8a46a3a8786f00a5102ef8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:44.962671 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c64507-db7b-478b-9911-aaa7fb01f799 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:44.963164 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c64507-db7b-478b-9911-aaa7fb01f799 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:44.963374 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c64507-db7b-478b-9911-aaa7fb01f799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:44.983361 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e3c64507-db7b-478b-9911-aaa7fb01f799 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:44.984564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e3c64507-db7b-478b-9911-aaa7fb01f799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:45.047600 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1250/2504] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:45.056124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:45.057648 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:45.119051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1f578-8a2f-421d-aa27-04b1bdf89076 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:45.122679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23c1e3-95e0-4313-8732-729826040f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:45.123287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23c1e3-95e0-4313-8732-729826040f72 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:45.123563 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23c1e3-95e0-4313-8732-729826040f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:45.123832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23c1e3-95e0-4313-8732-729826040f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:45.132865 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23c1e3-95e0-4313-8732-729826040f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:45.134315 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23c1e3-95e0-4313-8732-729826040f72 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:45.143812 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1255/2505] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:45.424137 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c64507-db7b-478b-9911-aaa7fb01f799 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:45.425771 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e3c64507-db7b-478b-9911-aaa7fb01f799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:45.446231 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c64507-db7b-478b-9911-aaa7fb01f799 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:45.462297 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:45.462970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:45.463408 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:45.463826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] PATH_INFO: `/v3/projects/94039ca1ad9a49bcae0e560a242c42f2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:45.465187 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] RBAC: Authorizing `identity:delete_project(project_id=94039ca1ad9a49bcae0e560a242c42f2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:45.467427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:45.501022 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:45.523644 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94039ca1ad9a49bcae0e560a242c42f2', 'request_id': 'req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.543748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94039ca1ad9a49bcae0e560a242c42f2', 'request_id': 'req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.552863 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94039ca1ad9a49bcae0e560a242c42f2', 'request_id': 'req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.566960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] The token cache is being invalidate because project 94039ca1ad9a49bcae0e560a242c42f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:39:45.567285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.568660 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.570784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1251/2506] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:39:45 2022] DELETE /identity/v3/projects/94039ca1ad9a49bcae0e560a242c42f2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:45.575008 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23c1e3-95e0-4313-8732-729826040f72 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:45.585794 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23c1e3-95e0-4313-8732-729826040f72 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:45.652200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:45.653841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:45.654350 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:45.654701 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] PATH_INFO: `/v3/users/71664ecc4360430096e45658ad4dfff4` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:45.655945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=71664ecc4360430096e45658ad4dfff4)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:45.658214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:45.719067 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71664ecc4360430096e45658ad4dfff4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.720280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71664ecc4360430096e45658ad4dfff4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.734073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71664ecc4360430096e45658ad4dfff4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.736143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:45.742672 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71664ecc4360430096e45658ad4dfff4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.743508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71664ecc4360430096e45658ad4dfff4'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:45.761428 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1256/2507] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:39:45 2022] DELETE /identity/v3/users/71664ecc4360430096e45658ad4dfff4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:46.062849 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:46.066452 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:46.086047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f97a5-5aa2-4b04-9f80-cf5fb56b94de admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:46.146592 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:46.147268 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:46.147549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:46.147826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] PATH_INFO: `/v3/projects/0523235af50f4f01a6d4d11fd5b61e19` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:46.148986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0523235af50f4f01a6d4d11fd5b61e19)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:46.150955 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:46.202558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0523235af50f4f01a6d4d11fd5b61e19', 'request_id': 'req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.208935 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0523235af50f4f01a6d4d11fd5b61e19', 'request_id': 'req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.219468 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0523235af50f4f01a6d4d11fd5b61e19', 'request_id': 'req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.232153 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] The token cache is being invalidate because project 0523235af50f4f01a6d4d11fd5b61e19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:39:46.232430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.233099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.235048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1252/2508] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:39:46 2022] DELETE /identity/v3/projects/0523235af50f4f01a6d4d11fd5b61e19 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:46.238992 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:46.240973 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:46.254331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bccc7-4043-42f4-9823-1e054c5b9aeb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:46.329073 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:46.329774 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:46.330054 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:46.330370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] PATH_INFO: `/v3/users/71c21fd2849a4b408d3104bb16211648` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:46.331526 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c21fd2849a4b408d3104bb16211648)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:46.333511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:46.386300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c21fd2849a4b408d3104bb16211648'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.386964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c21fd2849a4b408d3104bb16211648'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.399578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c21fd2849a4b408d3104bb16211648'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.408206 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c21fd2849a4b408d3104bb16211648'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.409254 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2b462fbc-3359-4c0b-8bed-0119c66f1054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c21fd2849a4b408d3104bb16211648'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:46.431292 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1257/2509] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:39:46 2022] DELETE /identity/v3/users/71c21fd2849a4b408d3104bb16211648 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:47.012220 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.013957 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:47.029282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8b76d-05d5-4a7a-9bd1-8b85c72d8ea3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.082353 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:47.102631 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.103541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.104024 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.104484 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] PATH_INFO: `/v3/projects/195680ced40a4136adfd43ade8b55734` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.106061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=195680ced40a4136adfd43ade8b55734)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.108256 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.159416 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195680ced40a4136adfd43ade8b55734', 'request_id': 'req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:47.165948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195680ced40a4136adfd43ade8b55734', 'request_id': 'req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:47.174803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195680ced40a4136adfd43ade8b55734', 'request_id': 'req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:47.185987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] The token cache is being invalidate because project 195680ced40a4136adfd43ade8b55734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:39:47.186367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:47.186919 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6949eec4-e86b-4567-bbc8-50eeba0c9dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:47.188412 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1253/2510] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:39:47 2022] DELETE /identity/v3/projects/195680ced40a4136adfd43ade8b55734 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:47.195207 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2bc58-69b4-4977-80e5-15e16bc3a042 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.195642 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2bc58-69b4-4977-80e5-15e16bc3a042 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.196002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2bc58-69b4-4977-80e5-15e16bc3a042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.215789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eca2bc58-69b4-4977-80e5-15e16bc3a042 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:47.217084 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eca2bc58-69b4-4977-80e5-15e16bc3a042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:47.278798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1258/2511] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:39:47.289017 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9477894-77c7-40e4-a545-54adf4fb88e6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.289364 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9477894-77c7-40e4-a545-54adf4fb88e6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.289581 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9477894-77c7-40e4-a545-54adf4fb88e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.312097 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c9477894-77c7-40e4-a545-54adf4fb88e6 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:47.313309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9477894-77c7-40e4-a545-54adf4fb88e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:47.321483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1254/2512] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:47.325842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2bc58-69b4-4977-80e5-15e16bc3a042 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.332541 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:47.336710 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2bc58-69b4-4977-80e5-15e16bc3a042 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.343363 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.343990 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.344523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.344960 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.345666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:39:47.346579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.348231 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.361113 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1259/2513] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:39:47.365722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9477894-77c7-40e4-a545-54adf4fb88e6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.375691 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9477894-77c7-40e4-a545-54adf4fb88e6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.380900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd20a4b-4e55-45ca-ae51-75eaa1b35c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.381498 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20a4b-4e55-45ca-ae51-75eaa1b35c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.381785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20a4b-4e55-45ca-ae51-75eaa1b35c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.382057 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20a4b-4e55-45ca-ae51-75eaa1b35c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.382748 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd20a4b-4e55-45ca-ae51-75eaa1b35c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.384798 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd20a4b-4e55-45ca-ae51-75eaa1b35c7e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.417661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1255/2514] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:39:47 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:47.422262 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.432152 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802828be-e5fc-4bd5-83e1-80b2191a9f49 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.491111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df752b-21c0-4b02-a09d-ac11be7f63d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.491717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df752b-21c0-4b02-a09d-ac11be7f63d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.492001 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df752b-21c0-4b02-a09d-ac11be7f63d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.492257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df752b-21c0-4b02-a09d-ac11be7f63d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.492883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df752b-21c0-4b02-a09d-ac11be7f63d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.494958 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df752b-21c0-4b02-a09d-ac11be7f63d6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.521789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1260/2515] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:39:47 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:47.526203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd20a4b-4e55-45ca-ae51-75eaa1b35c7e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.537074 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd20a4b-4e55-45ca-ae51-75eaa1b35c7e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.542651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661f356-1997-4242-8f76-c31c4a513b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.543265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f356-1997-4242-8f76-c31c4a513b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.543544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f356-1997-4242-8f76-c31c4a513b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.543813 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f356-1997-4242-8f76-c31c4a513b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.544458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661f356-1997-4242-8f76-c31c4a513b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.546106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661f356-1997-4242-8f76-c31c4a513b64 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.557720 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1256/2516] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:47.563036 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df752b-21c0-4b02-a09d-ac11be7f63d6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.574579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df752b-21c0-4b02-a09d-ac11be7f63d6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.580662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe494b41-56b8-4719-a79c-71be68e65b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.581346 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe494b41-56b8-4719-a79c-71be68e65b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.581706 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe494b41-56b8-4719-a79c-71be68e65b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.582043 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe494b41-56b8-4719-a79c-71be68e65b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.582835 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe494b41-56b8-4719-a79c-71be68e65b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.584498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe494b41-56b8-4719-a79c-71be68e65b11 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.596039 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1261/2517] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:39:47.600699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661f356-1997-4242-8f76-c31c4a513b64 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.610929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661f356-1997-4242-8f76-c31c4a513b64 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.616679 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b401c-42b8-41a3-b5ff-8f11892203ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.617335 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b401c-42b8-41a3-b5ff-8f11892203ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.617742 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b401c-42b8-41a3-b5ff-8f11892203ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.618012 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b401c-42b8-41a3-b5ff-8f11892203ba admin admin] PATH_INFO: `/v3/projects/e127e582231945be8c76ec48d10bca1a/users/6c0b1ab5761a4ad3adef2f79d62cb40c/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.634676 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b401c-42b8-41a3-b5ff-8f11892203ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e127e582231945be8c76ec48d10bca1a, user_id=6c0b1ab5761a4ad3adef2f79d62cb40c, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.637323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b401c-42b8-41a3-b5ff-8f11892203ba admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.653417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1257/2518] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:39:47 2022] PUT /identity/v3/projects/e127e582231945be8c76ec48d10bca1a/users/6c0b1ab5761a4ad3adef2f79d62cb40c/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:47.670878 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e61843-2269-4b34-9182-a37d13255e6c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.671174 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e61843-2269-4b34-9182-a37d13255e6c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.671499 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e61843-2269-4b34-9182-a37d13255e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.709872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-06e61843-2269-4b34-9182-a37d13255e6c None None] MFA Rules not processed for user `6c0b1ab5761a4ad3adef2f79d62cb40c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:47.711225 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-06e61843-2269-4b34-9182-a37d13255e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:47.785032 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1262/2519] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:47.790504 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b401c-42b8-41a3-b5ff-8f11892203ba admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.856061 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b401c-42b8-41a3-b5ff-8f11892203ba admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.861133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2b76-72ed-475f-8a75-3a380e5b535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.861833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2b76-72ed-475f-8a75-3a380e5b535e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.862159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2b76-72ed-475f-8a75-3a380e5b535e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.862448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2b76-72ed-475f-8a75-3a380e5b535e admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.863081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc2b76-72ed-475f-8a75-3a380e5b535e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.865127 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc2b76-72ed-475f-8a75-3a380e5b535e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.886683 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:47.903047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1258/2520] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:39:47 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:47.907347 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e61843-2269-4b34-9182-a37d13255e6c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:47.918800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e61843-2269-4b34-9182-a37d13255e6c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:47.979336 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb25a93-2de7-47dd-ac02-d8af356e1122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:47.979937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb25a93-2de7-47dd-ac02-d8af356e1122 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:47.980214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb25a93-2de7-47dd-ac02-d8af356e1122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:47.980482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb25a93-2de7-47dd-ac02-d8af356e1122 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:47.981102 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb25a93-2de7-47dd-ac02-d8af356e1122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:47.983128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb25a93-2de7-47dd-ac02-d8af356e1122 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:47.991230 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:48.007598 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1263/2521] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:39:47 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:48.011655 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2b76-72ed-475f-8a75-3a380e5b535e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:48.022563 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2b76-72ed-475f-8a75-3a380e5b535e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:48.028016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfd425-22d4-44e0-91f2-495c278600d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:48.028612 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfd425-22d4-44e0-91f2-495c278600d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:48.028886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfd425-22d4-44e0-91f2-495c278600d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:48.029152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfd425-22d4-44e0-91f2-495c278600d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:48.029794 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfd425-22d4-44e0-91f2-495c278600d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:48.031441 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfd425-22d4-44e0-91f2-495c278600d1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:48.042082 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1259/2522] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:48.046199 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb25a93-2de7-47dd-ac02-d8af356e1122 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:48.057011 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb25a93-2de7-47dd-ac02-d8af356e1122 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:48.062370 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32796019-28c5-4d3e-a669-cf44f544bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:48.063020 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32796019-28c5-4d3e-a669-cf44f544bff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:48.063304 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32796019-28c5-4d3e-a669-cf44f544bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:48.063579 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32796019-28c5-4d3e-a669-cf44f544bff8 admin admin] PATH_INFO: `/v3/projects/c9c940b40ff046fcae3faf51aa103b40/users/0d3581f8ad5640999ff35e2f8b047674/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:48.079129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32796019-28c5-4d3e-a669-cf44f544bff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c940b40ff046fcae3faf51aa103b40, user_id=0d3581f8ad5640999ff35e2f8b047674, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:48.081822 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32796019-28c5-4d3e-a669-cf44f544bff8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:48.096594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1264/2523] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:39:48 2022] PUT /identity/v3/projects/c9c940b40ff046fcae3faf51aa103b40/users/0d3581f8ad5640999ff35e2f8b047674/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:48.100900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfd425-22d4-44e0-91f2-495c278600d1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:48.111216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfd425-22d4-44e0-91f2-495c278600d1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:48.168930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763aa66-5724-4aeb-a43c-0fad2df5ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:48.169528 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763aa66-5724-4aeb-a43c-0fad2df5ecd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:48.169784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763aa66-5724-4aeb-a43c-0fad2df5ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:48.170050 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763aa66-5724-4aeb-a43c-0fad2df5ecd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:48.170714 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763aa66-5724-4aeb-a43c-0fad2df5ecd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:48.172291 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763aa66-5724-4aeb-a43c-0fad2df5ecd9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:48.182440 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1260/2524] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:48.187146 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32796019-28c5-4d3e-a669-cf44f544bff8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:48.198568 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32796019-28c5-4d3e-a669-cf44f544bff8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:48.205033 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:48.205721 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:48.205995 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:48.206300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] PATH_INFO: `/v3/projects/c9c940b40ff046fcae3faf51aa103b40/users/0d3581f8ad5640999ff35e2f8b047674/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:48.208565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c940b40ff046fcae3faf51aa103b40, user_id=0d3581f8ad5640999ff35e2f8b047674, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:48.211253 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:48.225872 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1265/2525] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:39:48 2022] PUT /identity/v3/projects/c9c940b40ff046fcae3faf51aa103b40/users/0d3581f8ad5640999ff35e2f8b047674/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:48.236956 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfdcf9-18b0-4652-a4c7-f04a84258e0f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:48.237323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfdcf9-18b0-4652-a4c7-f04a84258e0f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:48.237625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfdcf9-18b0-4652-a4c7-f04a84258e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:48.270597 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c0cfdcf9-18b0-4652-a4c7-f04a84258e0f None None] MFA Rules not processed for user `0d3581f8ad5640999ff35e2f8b047674`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:48.271793 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c0cfdcf9-18b0-4652-a4c7-f04a84258e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:48.339221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1261/2526] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:48.353196 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:48.355260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:48.418387 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cf161-6fcd-4694-8d35-8ffdc9fc1a34 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:48.422661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9acba-9ad5-495a-9af6-79957bd4e72a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:48.423249 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9acba-9ad5-495a-9af6-79957bd4e72a service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:48.423522 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9acba-9ad5-495a-9af6-79957bd4e72a service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:48.423797 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9acba-9ad5-495a-9af6-79957bd4e72a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:48.433593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9acba-9ad5-495a-9af6-79957bd4e72a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:48.435053 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9acba-9ad5-495a-9af6-79957bd4e72a service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:48.445245 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1266/2527] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:48 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:48.492857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfdcf9-18b0-4652-a4c7-f04a84258e0f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:48.493755 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c0cfdcf9-18b0-4652-a4c7-f04a84258e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:48.506615 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfdcf9-18b0-4652-a4c7-f04a84258e0f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:48.510706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8ce26-f3e0-4800-bccb-186d57f7821b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:48.511313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ce26-f3e0-4800-bccb-186d57f7821b service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:48.511589 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ce26-f3e0-4800-bccb-186d57f7821b service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:48.511857 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8ce26-f3e0-4800-bccb-186d57f7821b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:48.570694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8ce26-f3e0-4800-bccb-186d57f7821b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:48.572434 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8ce26-f3e0-4800-bccb-186d57f7821b service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:48.581785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1262/2528] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:52.011338 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9acba-9ad5-495a-9af6-79957bd4e72a service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:52.075697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9acba-9ad5-495a-9af6-79957bd4e72a service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:52.080493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:52.081023 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:52.081306 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:52.081581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] PATH_INFO: `/v3/users/6c0b1ab5761a4ad3adef2f79d62cb40c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:52.082860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c0b1ab5761a4ad3adef2f79d62cb40c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:52.084764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:52.135088 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c0b1ab5761a4ad3adef2f79d62cb40c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.135870 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c0b1ab5761a4ad3adef2f79d62cb40c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.143025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:52.150694 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c0b1ab5761a4ad3adef2f79d62cb40c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.157687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c0b1ab5761a4ad3adef2f79d62cb40c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.158406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c0b1ab5761a4ad3adef2f79d62cb40c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.172130 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1267/2529] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:39:52 2022] DELETE /identity/v3/users/6c0b1ab5761a4ad3adef2f79d62cb40c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:52.176558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1d1c-e40b-4787-8069-644420a970d3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:52.177427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1d1c-e40b-4787-8069-644420a970d3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:52.177713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1d1c-e40b-4787-8069-644420a970d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:52.196400 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:52.203777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f44a1d1c-e40b-4787-8069-644420a970d3 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:52.204947 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f44a1d1c-e40b-4787-8069-644420a970d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:52.266950 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1263/2530] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:52.277212 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:52.278912 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:52.342494 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c76c5-68d7-4261-8d17-8eda7d3f9302 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:52.346938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:52.347549 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:52.347817 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:52.348082 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:52.358031 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:52.359507 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:52.369400 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1268/2531] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:52.637568 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1d1c-e40b-4787-8069-644420a970d3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:52.639602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f44a1d1c-e40b-4787-8069-644420a970d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:52.672493 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:52.674384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:52.696380 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24eef-f44c-4a00-8b40-8dc72481e9c4 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:52.706025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:52.706769 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:52.707048 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:52.707606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] PATH_INFO: `/v3/projects/e127e582231945be8c76ec48d10bca1a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:52.708686 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e127e582231945be8c76ec48d10bca1a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:52.710765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:52.740878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1d1c-e40b-4787-8069-644420a970d3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:52.756122 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:52.756862 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:52.757205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:52.757540 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:52.759975 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:52.773282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:52.775009 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:52.787169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1264/2532] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:52.803228 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e127e582231945be8c76ec48d10bca1a', 'request_id': 'req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.814368 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e127e582231945be8c76ec48d10bca1a', 'request_id': 'req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.822798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e127e582231945be8c76ec48d10bca1a', 'request_id': 'req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.834637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] The token cache is being invalidate because project e127e582231945be8c76ec48d10bca1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:39:52.834927 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.835594 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.837309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1269/2533] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:39:52 2022] DELETE /identity/v3/projects/e127e582231945be8c76ec48d10bca1a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:39:52.841373 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:52.852203 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418868d-fd3c-4d1f-a0e0-5b87e9c3de0d service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:52.915047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:52.915047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:52.915047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:52.915047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] PATH_INFO: `/v3/users/0d3581f8ad5640999ff35e2f8b047674` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:52.915459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3581f8ad5640999ff35e2f8b047674)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:52.915459 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:52.958402 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:52.969579 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3581f8ad5640999ff35e2f8b047674'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.970384 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3581f8ad5640999ff35e2f8b047674'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.983542 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3581f8ad5640999ff35e2f8b047674'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.991722 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3581f8ad5640999ff35e2f8b047674'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:52.992827 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5498e883-27f6-43b9-baf4-84c38b6afe7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3581f8ad5640999ff35e2f8b047674'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:53.009843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1265/2534] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:39:52 2022] DELETE /identity/v3/users/0d3581f8ad5640999ff35e2f8b047674 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:53.315783 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:53.317362 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:53.330796 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a0c2e-a969-4f3d-bcb6-6fd5f560bc10 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:53.395385 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:53.396165 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.396513 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.396853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] PATH_INFO: `/v3/projects/c9c940b40ff046fcae3faf51aa103b40` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.398112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c940b40ff046fcae3faf51aa103b40)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:53.399910 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:53.401464 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:53.450122 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c940b40ff046fcae3faf51aa103b40', 'request_id': 'req-2f6d7221-8f13-46ae-b03f-382d9991aaac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:53.459807 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c940b40ff046fcae3faf51aa103b40', 'request_id': 'req-2f6d7221-8f13-46ae-b03f-382d9991aaac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:53.466255 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c940b40ff046fcae3faf51aa103b40', 'request_id': 'req-2f6d7221-8f13-46ae-b03f-382d9991aaac'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:53.477475 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] The token cache is being invalidate because project c9c940b40ff046fcae3faf51aa103b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:39:53.477759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:53.478442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2f6d7221-8f13-46ae-b03f-382d9991aaac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:53.479944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1270/2535] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:39:53 2022] DELETE /identity/v3/projects/c9c940b40ff046fcae3faf51aa103b40 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:53.484999 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bdcc-9722-46cc-89a7-ec2e7a9b074d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.485494 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bdcc-9722-46cc-89a7-ec2e7a9b074d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.485593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bdcc-9722-46cc-89a7-ec2e7a9b074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.506916 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a853bdcc-9722-46cc-89a7-ec2e7a9b074d None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:53.508134 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a853bdcc-9722-46cc-89a7-ec2e7a9b074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:53.569694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1266/2536] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:53.635482 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e974-bcd8-495f-8049-56f993174ef5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.635735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e974-bcd8-495f-8049-56f993174ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.636014 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e974-bcd8-495f-8049-56f993174ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.657248 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1d38e974-bcd8-495f-8049-56f993174ef5 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:53.658468 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1d38e974-bcd8-495f-8049-56f993174ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:53.666711 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1271/2537] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:53.671270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853bdcc-9722-46cc-89a7-ec2e7a9b074d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:53.682016 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853bdcc-9722-46cc-89a7-ec2e7a9b074d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:53.687159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:53.687752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.688040 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.688311 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.688960 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:39:53.689260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:53.690891 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:53.703133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1267/2538] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:39:53.707862 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38e974-bcd8-495f-8049-56f993174ef5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:53.717708 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38e974-bcd8-495f-8049-56f993174ef5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:53.723506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70471dd9-e64e-47a7-94d8-7e4e02d89832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:53.724112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471dd9-e64e-47a7-94d8-7e4e02d89832 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.724395 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471dd9-e64e-47a7-94d8-7e4e02d89832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.724689 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471dd9-e64e-47a7-94d8-7e4e02d89832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.725327 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70471dd9-e64e-47a7-94d8-7e4e02d89832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:53.727275 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70471dd9-e64e-47a7-94d8-7e4e02d89832 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:53.757120 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1272/2539] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:39:53 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:53.761558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:53.772014 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a191d-01be-4a6f-b765-922e0e3dcca7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:53.827770 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e26aab-d89b-4e58-945d-b2b91c80fa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:53.828371 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e26aab-d89b-4e58-945d-b2b91c80fa9f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.828625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e26aab-d89b-4e58-945d-b2b91c80fa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.828930 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e26aab-d89b-4e58-945d-b2b91c80fa9f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.829558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e26aab-d89b-4e58-945d-b2b91c80fa9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:53.831516 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e26aab-d89b-4e58-945d-b2b91c80fa9f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:53.858508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1268/2540] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:39:53 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:39:53.862645 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70471dd9-e64e-47a7-94d8-7e4e02d89832 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:53.874139 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70471dd9-e64e-47a7-94d8-7e4e02d89832 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:53.878855 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ca337-5ac9-4cba-8e2d-090f308747db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:53.880924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca337-5ac9-4cba-8e2d-090f308747db admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.880924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca337-5ac9-4cba-8e2d-090f308747db admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.880924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca337-5ac9-4cba-8e2d-090f308747db admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.881260 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ca337-5ac9-4cba-8e2d-090f308747db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:53.882861 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ca337-5ac9-4cba-8e2d-090f308747db admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:53.893398 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1273/2541] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:53.898409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e26aab-d89b-4e58-945d-b2b91c80fa9f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:53.909558 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e26aab-d89b-4e58-945d-b2b91c80fa9f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:53.919785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f2053-8684-4c43-9ff9-637931ef00f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:53.920375 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2053-8684-4c43-9ff9-637931ef00f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.920657 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2053-8684-4c43-9ff9-637931ef00f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.921090 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f2053-8684-4c43-9ff9-637931ef00f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.921586 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f2053-8684-4c43-9ff9-637931ef00f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:53.923226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f2053-8684-4c43-9ff9-637931ef00f7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:53.935711 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1269/2542] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:39:53.939996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ca337-5ac9-4cba-8e2d-090f308747db admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:53.950607 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ca337-5ac9-4cba-8e2d-090f308747db admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:53.955129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:53.955789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:53.956311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:53.956311 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] PATH_INFO: `/v3/projects/e84ba2470fe6453cbbac2f72b9a87e29/users/b6a1768828944756a167710adfc2d3dc/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:53.972402 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84ba2470fe6453cbbac2f72b9a87e29, user_id=b6a1768828944756a167710adfc2d3dc, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:53.975704 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:53.989899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1274/2543] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:39:53 2022] PUT /identity/v3/projects/e84ba2470fe6453cbbac2f72b9a87e29/users/b6a1768828944756a167710adfc2d3dc/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:39:54.000470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603230-287e-45e1-85f4-d2766f1eac85 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:54.000763 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603230-287e-45e1-85f4-d2766f1eac85 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:54.001041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603230-287e-45e1-85f4-d2766f1eac85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:54.032968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a5603230-287e-45e1-85f4-d2766f1eac85 None None] MFA Rules not processed for user `b6a1768828944756a167710adfc2d3dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:39:54.032968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a5603230-287e-45e1-85f4-d2766f1eac85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:54.036202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:54.097870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1270/2544] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:39:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:39:54.109759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:54.111627 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:54.196331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c004f6d-ff53-4501-a0bc-deb8b6ebce84 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:54.200286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25900be-8c71-4858-ad95-f57149f7d18a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fd09151e284f639c93699b0d26db05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:54.200889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25900be-8c71-4858-ad95-f57149f7d18a service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:54.201167 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25900be-8c71-4858-ad95-f57149f7d18a service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:54.201442 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25900be-8c71-4858-ad95-f57149f7d18a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:54.210215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25900be-8c71-4858-ad95-f57149f7d18a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:54.211653 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25900be-8c71-4858-ad95-f57149f7d18a service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:54.220637 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1275/2545] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:40:04.094379 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=24 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:04.216753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:11.783784 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5603230-287e-45e1-85f4-d2766f1eac85 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:11.785461 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a5603230-287e-45e1-85f4-d2766f1eac85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:11.845201 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5603230-287e-45e1-85f4-d2766f1eac85 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:11.851314 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e9dc7-8a85-4f0d-b47d-0367483e288e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:11.851932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9dc7-8a85-4f0d-b47d-0367483e288e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:11.852213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9dc7-8a85-4f0d-b47d-0367483e288e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:11.852479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9dc7-8a85-4f0d-b47d-0367483e288e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:11.854738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a08e9dc7-8a85-4f0d-b47d-0367483e288e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:11.934939 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e9dc7-8a85-4f0d-b47d-0367483e288e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:11.936401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e9dc7-8a85-4f0d-b47d-0367483e288e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:11.946522 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1271/2546] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:40:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:40:21.941870 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=50 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:23.648495 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25900be-8c71-4858-ad95-f57149f7d18a service glance] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:23.715751 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25900be-8c71-4858-ad95-f57149f7d18a service glance] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:23.720582 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:23.721565 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:23.721834 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:23.722105 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] PATH_INFO: `/v3/users/b6a1768828944756a167710adfc2d3dc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:23.723232 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a1768828944756a167710adfc2d3dc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:23.725127 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:23.774445 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a1768828944756a167710adfc2d3dc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:23.775153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a1768828944756a167710adfc2d3dc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:23.787293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a1768828944756a167710adfc2d3dc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:23.794483 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a1768828944756a167710adfc2d3dc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:23.795129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a1768828944756a167710adfc2d3dc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:23.810481 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1276/2547] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:40:23 2022] DELETE /identity/v3/users/b6a1768828944756a167710adfc2d3dc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:40:23.815330 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979cd6f-c8e5-4f62-91ef-36ef5dbe873c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:23.815837 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979cd6f-c8e5-4f62-91ef-36ef5dbe873c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:23.816013 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979cd6f-c8e5-4f62-91ef-36ef5dbe873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:23.835875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0979cd6f-c8e5-4f62-91ef-36ef5dbe873c None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:40:23.837067 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0979cd6f-c8e5-4f62-91ef-36ef5dbe873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:23.904026 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1272/2548] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:40:23.914062 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:23.915978 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:23.977124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f40755-368d-4a0d-a630-0ff932ac9856 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:23.981429 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1814b3-ac35-47cb-8bf5-f34cdc199679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:23.982035 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1814b3-ac35-47cb-8bf5-f34cdc199679 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:23.982344 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1814b3-ac35-47cb-8bf5-f34cdc199679 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:23.982601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1814b3-ac35-47cb-8bf5-f34cdc199679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:23.991935 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1814b3-ac35-47cb-8bf5-f34cdc199679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:23.993358 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1814b3-ac35-47cb-8bf5-f34cdc199679 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:24.004187 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1277/2549] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:40:24.299881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979cd6f-c8e5-4f62-91ef-36ef5dbe873c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:24.301401 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0979cd6f-c8e5-4f62-91ef-36ef5dbe873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:24.313678 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979cd6f-c8e5-4f62-91ef-36ef5dbe873c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:24.319830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:24.320500 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.320778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.321105 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] PATH_INFO: `/v3/projects/e84ba2470fe6453cbbac2f72b9a87e29` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.322207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84ba2470fe6453cbbac2f72b9a87e29)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:24.325662 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:24.375532 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84ba2470fe6453cbbac2f72b9a87e29', 'request_id': 'req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:24.382037 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84ba2470fe6453cbbac2f72b9a87e29', 'request_id': 'req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:24.391984 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84ba2470fe6453cbbac2f72b9a87e29', 'request_id': 'req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:24.403008 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] The token cache is being invalidate because project e84ba2470fe6453cbbac2f72b9a87e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:40:24.403280 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:24.403904 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7f787091-2b58-4aa0-bd4e-f1ffdf926cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:24.405417 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1273/2550] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:40:24 2022] DELETE /identity/v3/projects/e84ba2470fe6453cbbac2f72b9a87e29 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:40:24.410687 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbf8db-2057-4073-8dea-559f5a25ba6f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.410965 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbf8db-2057-4073-8dea-559f5a25ba6f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.411239 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbf8db-2057-4073-8dea-559f5a25ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.432703 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-86fbf8db-2057-4073-8dea-559f5a25ba6f None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:40:24.433895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-86fbf8db-2057-4073-8dea-559f5a25ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:24.452508 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:24.498224 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1278/2551] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:40:24.508136 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c274f1-5455-435b-bc4a-393eaa6f19b4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.508460 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c274f1-5455-435b-bc4a-393eaa6f19b4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.508803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c274f1-5455-435b-bc4a-393eaa6f19b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.534808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-86c274f1-5455-435b-bc4a-393eaa6f19b4 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:40:24.536007 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-86c274f1-5455-435b-bc4a-393eaa6f19b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:24.544483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1274/2552] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:40:24.548864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbf8db-2057-4073-8dea-559f5a25ba6f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:24.559051 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbf8db-2057-4073-8dea-559f5a25ba6f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:24.563833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:24.564431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.564735 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.565002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.565649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:40:24.565950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:24.567589 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:24.580474 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1279/2553] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:40:24.584936 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c274f1-5455-435b-bc4a-393eaa6f19b4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:24.594708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c274f1-5455-435b-bc4a-393eaa6f19b4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:24.600169 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5faf2-0f4a-45c6-9636-959f4b06fb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:24.600757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5faf2-0f4a-45c6-9636-959f4b06fb74 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.601043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5faf2-0f4a-45c6-9636-959f4b06fb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.601312 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5faf2-0f4a-45c6-9636-959f4b06fb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.601940 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5faf2-0f4a-45c6-9636-959f4b06fb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:24.603922 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5faf2-0f4a-45c6-9636-959f4b06fb74 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:24.632938 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1275/2554] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:40:24 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:40:24.637285 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:24.647424 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd2c7-112c-4c24-87c6-371d4df54ccb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:24.704821 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9982fd-cd07-4285-b5d1-1ef318ba0f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:24.705438 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9982fd-cd07-4285-b5d1-1ef318ba0f34 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.705585 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9982fd-cd07-4285-b5d1-1ef318ba0f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.705884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9982fd-cd07-4285-b5d1-1ef318ba0f34 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.706532 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9982fd-cd07-4285-b5d1-1ef318ba0f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:24.708422 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9982fd-cd07-4285-b5d1-1ef318ba0f34 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:24.729467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1280/2555] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:40:24 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:40:24.733694 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5faf2-0f4a-45c6-9636-959f4b06fb74 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:24.744060 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5faf2-0f4a-45c6-9636-959f4b06fb74 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:24.748843 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d613c3-54e8-4228-83ef-7c3e04fc46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:24.749436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d613c3-54e8-4228-83ef-7c3e04fc46b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.749685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d613c3-54e8-4228-83ef-7c3e04fc46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.749981 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d613c3-54e8-4228-83ef-7c3e04fc46b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.750684 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d613c3-54e8-4228-83ef-7c3e04fc46b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:24.752279 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d613c3-54e8-4228-83ef-7c3e04fc46b3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:24.762564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1276/2556] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:40:24.766732 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9982fd-cd07-4285-b5d1-1ef318ba0f34 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:24.777651 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9982fd-cd07-4285-b5d1-1ef318ba0f34 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:24.782999 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe95535-c6f2-42e6-9346-537240f09d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:24.783647 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe95535-c6f2-42e6-9346-537240f09d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.783924 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe95535-c6f2-42e6-9346-537240f09d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.784215 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe95535-c6f2-42e6-9346-537240f09d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.784858 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe95535-c6f2-42e6-9346-537240f09d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:24.786498 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe95535-c6f2-42e6-9346-537240f09d6e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:24.799750 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1281/2557] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:40:24.804209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d613c3-54e8-4228-83ef-7c3e04fc46b3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:24.815560 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d613c3-54e8-4228-83ef-7c3e04fc46b3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:24.822257 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:24.822910 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.823185 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.823453 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] PATH_INFO: `/v3/projects/05896bf0624a494ab773e5c70cb6d87c/users/547018857dfe4b6f8c8263e970fc3878/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.842215 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05896bf0624a494ab773e5c70cb6d87c, user_id=547018857dfe4b6f8c8263e970fc3878, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:24.844888 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:24.860931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1277/2558] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:40:24 2022] PUT /identity/v3/projects/05896bf0624a494ab773e5c70cb6d87c/users/547018857dfe4b6f8c8263e970fc3878/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:40:24.872716 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2f95-c535-413e-81c7-a503cd8cade0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:24.873081 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2f95-c535-413e-81c7-a503cd8cade0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:24.873511 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2f95-c535-413e-81c7-a503cd8cade0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:24.904240 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eafe2f95-c535-413e-81c7-a503cd8cade0 None None] MFA Rules not processed for user `547018857dfe4b6f8c8263e970fc3878`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:40:24.905437 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eafe2f95-c535-413e-81c7-a503cd8cade0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:24.975129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1282/2559] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:40:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:40:24.986987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:24.988855 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:25.011778 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:25.051367 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9550efa-9d9c-442d-94a8-a2347571c550 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:25.055128 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6bddc-080e-4666-82ff-756223449b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:25.055789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6bddc-080e-4666-82ff-756223449b6b service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:25.056070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6bddc-080e-4666-82ff-756223449b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:25.056339 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6bddc-080e-4666-82ff-756223449b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:25.065294 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6bddc-080e-4666-82ff-756223449b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:25.066803 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6bddc-080e-4666-82ff-756223449b6b service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:25.077085 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1278/2560] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:40:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:40:34.972360 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:35.071878 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=17 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:55.027169 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe2f95-c535-413e-81c7-a503cd8cade0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:55.093798 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe2f95-c535-413e-81c7-a503cd8cade0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:55.099578 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:55.100218 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:55.100480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:55.100764 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] PATH_INFO: `/v3/users/547018857dfe4b6f8c8263e970fc3878` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:55.101988 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=547018857dfe4b6f8c8263e970fc3878)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:55.103983 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:55.154288 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547018857dfe4b6f8c8263e970fc3878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.155112 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547018857dfe4b6f8c8263e970fc3878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.167479 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547018857dfe4b6f8c8263e970fc3878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.174643 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547018857dfe4b6f8c8263e970fc3878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.175316 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547018857dfe4b6f8c8263e970fc3878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.189836 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1283/2561] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:40:55 2022] DELETE /identity/v3/users/547018857dfe4b6f8c8263e970fc3878 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:40:55.194687 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda1142-fa12-4257-a6d7-e4850b01f587 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:55.195175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda1142-fa12-4257-a6d7-e4850b01f587 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:55.195274 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda1142-fa12-4257-a6d7-e4850b01f587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:55.214304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aeda1142-fa12-4257-a6d7-e4850b01f587 None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:40:55.215508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aeda1142-fa12-4257-a6d7-e4850b01f587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:55.281593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1279/2562] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:40:55.291730 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:55.293497 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:55.355680 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1c171-b829-4fb6-b240-836a21cb7721 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:55.359695 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d4994-acc5-4b69-8d49-6746bb198a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:55.360302 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4994-acc5-4b69-8d49-6746bb198a99 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:55.360581 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4994-acc5-4b69-8d49-6746bb198a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:55.360850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4994-acc5-4b69-8d49-6746bb198a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:55.370677 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d4994-acc5-4b69-8d49-6746bb198a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:55.372143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d4994-acc5-4b69-8d49-6746bb198a99 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:55.382021 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1284/2563] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:40:55.662735 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda1142-fa12-4257-a6d7-e4850b01f587 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:55.664398 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aeda1142-fa12-4257-a6d7-e4850b01f587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:55.677292 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda1142-fa12-4257-a6d7-e4850b01f587 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:55.682159 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:55.682978 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:55.683258 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:55.683661 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] PATH_INFO: `/v3/projects/05896bf0624a494ab773e5c70cb6d87c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:55.685048 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05896bf0624a494ab773e5c70cb6d87c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:55.687173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:55.737893 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05896bf0624a494ab773e5c70cb6d87c', 'request_id': 'req-4e6f082b-a733-449e-9fae-9edc8d6ab62c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.744081 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05896bf0624a494ab773e5c70cb6d87c', 'request_id': 'req-4e6f082b-a733-449e-9fae-9edc8d6ab62c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.754076 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05896bf0624a494ab773e5c70cb6d87c', 'request_id': 'req-4e6f082b-a733-449e-9fae-9edc8d6ab62c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.764721 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] The token cache is being invalidate because project 05896bf0624a494ab773e5c70cb6d87c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:40:55.765068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.765724 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4e6f082b-a733-449e-9fae-9edc8d6ab62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:40:55.767285 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1280/2564] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:40:55 2022] DELETE /identity/v3/projects/05896bf0624a494ab773e5c70cb6d87c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:40:55.772529 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af21d8-c490-4ec7-b2dd-ce5bda851c8b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:55.772808 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af21d8-c490-4ec7-b2dd-ce5bda851c8b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:55.773077 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af21d8-c490-4ec7-b2dd-ce5bda851c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:55.792887 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-33af21d8-c490-4ec7-b2dd-ce5bda851c8b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:40:55.794111 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-33af21d8-c490-4ec7-b2dd-ce5bda851c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:55.812025 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:55.857517 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1285/2565] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:40:55.867300 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ee936-1120-48b5-8513-445805e58aad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:55.867677 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ee936-1120-48b5-8513-445805e58aad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:55.867929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ee936-1120-48b5-8513-445805e58aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:55.888921 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c11ee936-1120-48b5-8513-445805e58aad None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:40:55.890395 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c11ee936-1120-48b5-8513-445805e58aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:55.898767 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1281/2566] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:40:55.903775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af21d8-c490-4ec7-b2dd-ce5bda851c8b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:55.915979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af21d8-c490-4ec7-b2dd-ce5bda851c8b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:55.923279 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:55.923883 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:55.924154 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:55.924418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:55.925086 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:40:55.925384 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:55.927002 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:55.940443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1286/2567] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Nov 2 07:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:40:55.945291 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ee936-1120-48b5-8513-445805e58aad None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:55.955518 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ee936-1120-48b5-8513-445805e58aad None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:55.961209 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b33d-3726-42f2-a3b3-4e7eac121817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:55.961844 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b33d-3726-42f2-a3b3-4e7eac121817 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:55.962143 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b33d-3726-42f2-a3b3-4e7eac121817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:55.962423 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b33d-3726-42f2-a3b3-4e7eac121817 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:55.963059 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596b33d-3726-42f2-a3b3-4e7eac121817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:55.964970 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596b33d-3726-42f2-a3b3-4e7eac121817 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:55.999202 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1282/2568] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:40:55 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:40:56.003587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:56.013852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60572321-0c84-490a-a1fc-4a8e76506c59 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:56.073626 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4da021-0747-415f-800f-a3479ed42555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:56.074309 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4da021-0747-415f-800f-a3479ed42555 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:56.074556 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4da021-0747-415f-800f-a3479ed42555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:56.074853 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4da021-0747-415f-800f-a3479ed42555 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:56.075515 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4da021-0747-415f-800f-a3479ed42555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:56.077420 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4da021-0747-415f-800f-a3479ed42555 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:56.101864 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1287/2569] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:40:56 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:40:56.106405 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b33d-3726-42f2-a3b3-4e7eac121817 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:56.117351 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b33d-3726-42f2-a3b3-4e7eac121817 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:56.123327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ef509-f0e0-4469-a4d6-dafb60572780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:56.123929 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ef509-f0e0-4469-a4d6-dafb60572780 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:56.124288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ef509-f0e0-4469-a4d6-dafb60572780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:56.124564 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ef509-f0e0-4469-a4d6-dafb60572780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:56.125245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ef509-f0e0-4469-a4d6-dafb60572780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:56.126908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ef509-f0e0-4469-a4d6-dafb60572780 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:56.137270 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1283/2570] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:40:56.141286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4da021-0747-415f-800f-a3479ed42555 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:56.151723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4da021-0747-415f-800f-a3479ed42555 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:56.157573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d5528-4415-4644-89dd-727ca4c8dbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:56.158190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d5528-4415-4644-89dd-727ca4c8dbed admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:56.158467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d5528-4415-4644-89dd-727ca4c8dbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:56.158758 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d5528-4415-4644-89dd-727ca4c8dbed admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:56.159391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d5528-4415-4644-89dd-727ca4c8dbed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:56.160986 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d5528-4415-4644-89dd-727ca4c8dbed admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:56.173192 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1288/2571] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:40:56.177410 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ef509-f0e0-4469-a4d6-dafb60572780 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:56.188192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ef509-f0e0-4469-a4d6-dafb60572780 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:56.193832 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:56.194515 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:56.194816 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:56.195386 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] PATH_INFO: `/v3/projects/5d00af8d4afc4004aa95bd16a703c148/users/f73df5ee33b14900bc8cbaa35275ac48/roles/2e66bd66105e4a1e9dcbda045dc2d057` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:56.211664 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d00af8d4afc4004aa95bd16a703c148, user_id=f73df5ee33b14900bc8cbaa35275ac48, role_id=2e66bd66105e4a1e9dcbda045dc2d057)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:56.216047 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:56.235129 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1284/2572] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:40:56 2022] PUT /identity/v3/projects/5d00af8d4afc4004aa95bd16a703c148/users/f73df5ee33b14900bc8cbaa35275ac48/roles/2e66bd66105e4a1e9dcbda045dc2d057 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:40:56.242731 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f28e2-ada2-476b-9f71-9398a2db686b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:56.243017 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f28e2-ada2-476b-9f71-9398a2db686b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:56.243343 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f28e2-ada2-476b-9f71-9398a2db686b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:56.266636 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a64f28e2-ada2-476b-9f71-9398a2db686b None None] MFA Rules not processed for user `085301de455d4179ae28198d5b0bd385`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:40:56.267778 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a64f28e2-ada2-476b-9f71-9398a2db686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:56.329596 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1289/2573] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:40:56.339578 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:40:56.341368 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:40:56.364871 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:40:56.403475 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8c36a-0201-469c-a497-4546769592d6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:40:56.407713 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd955a6-b22f-41c9-96a1-d88262cdedfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:40:56.408313 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd955a6-b22f-41c9-96a1-d88262cdedfb service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:40:56.408635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd955a6-b22f-41c9-96a1-d88262cdedfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:40:56.408908 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd955a6-b22f-41c9-96a1-d88262cdedfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:40:56.418427 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd955a6-b22f-41c9-96a1-d88262cdedfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:40:56.419905 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd955a6-b22f-41c9-96a1-d88262cdedfb service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:40:56.429548 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1285/2574] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:00.737746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d6a71-bd35-4b22-b61f-62be26ee420a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:00.738674 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d6a71-bd35-4b22-b61f-62be26ee420a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:00.739128 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d6a71-bd35-4b22-b61f-62be26ee420a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:00.772153 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ea4d6a71-bd35-4b22-b61f-62be26ee420a None None] MFA Rules not processed for user `f73df5ee33b14900bc8cbaa35275ac48`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:00.773505 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ea4d6a71-bd35-4b22-b61f-62be26ee420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:00.792250 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:00.846831 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1290/2575] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:41:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:00.851391 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd955a6-b22f-41c9-96a1-d88262cdedfb service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:00.862245 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd955a6-b22f-41c9-96a1-d88262cdedfb service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:00.868867 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505df6-91a3-4060-ac70-2cd68f4a806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:00.869719 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505df6-91a3-4060-ac70-2cd68f4a806e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:00.870208 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505df6-91a3-4060-ac70-2cd68f4a806e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:00.870650 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505df6-91a3-4060-ac70-2cd68f4a806e admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:00.871448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3505df6-91a3-4060-ac70-2cd68f4a806e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:00.873559 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3505df6-91a3-4060-ac70-2cd68f4a806e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:00.907436 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1286/2576] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Nov 2 07:41:00 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:00.914195 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d6a71-bd35-4b22-b61f-62be26ee420a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:00.925151 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d6a71-bd35-4b22-b61f-62be26ee420a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:00.990026 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6412f4a-dad2-472c-acc2-dcb13e82a0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:00.990666 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6412f4a-dad2-472c-acc2-dcb13e82a0bb admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:00.990941 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6412f4a-dad2-472c-acc2-dcb13e82a0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:00.991235 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6412f4a-dad2-472c-acc2-dcb13e82a0bb admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:00.991921 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6412f4a-dad2-472c-acc2-dcb13e82a0bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:00.993894 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6412f4a-dad2-472c-acc2-dcb13e82a0bb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:01.022655 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1291/2577] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Nov 2 07:41:00 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:41:01.023958 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505df6-91a3-4060-ac70-2cd68f4a806e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:01.038858 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505df6-91a3-4060-ac70-2cd68f4a806e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:01.045167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c4cca-024c-4930-b468-4bf6a8064c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:01.046727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4cca-024c-4930-b468-4bf6a8064c0b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:01.047029 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4cca-024c-4930-b468-4bf6a8064c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:01.047306 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4cca-024c-4930-b468-4bf6a8064c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:01.048011 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c4cca-024c-4930-b468-4bf6a8064c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:01.049707 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c4cca-024c-4930-b468-4bf6a8064c0b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:01.065987 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1287/2578] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:01.066512 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6412f4a-dad2-472c-acc2-dcb13e82a0bb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:01.079141 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6412f4a-dad2-472c-acc2-dcb13e82a0bb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:01.086004 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758a503-3f89-4202-9d52-696cfd30125d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:01.086776 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758a503-3f89-4202-9d52-696cfd30125d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:01.087123 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758a503-3f89-4202-9d52-696cfd30125d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:01.087460 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758a503-3f89-4202-9d52-696cfd30125d admin admin] PATH_INFO: `/v3/projects/02104a370e6243009c39ebf1979dad00/users/7c67911e2cf84f99a9e077e89cae272e/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:01.109222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758a503-3f89-4202-9d52-696cfd30125d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02104a370e6243009c39ebf1979dad00, user_id=7c67911e2cf84f99a9e077e89cae272e, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:01.113487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758a503-3f89-4202-9d52-696cfd30125d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:01.132847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1292/2579] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:01 2022] PUT /identity/v3/projects/02104a370e6243009c39ebf1979dad00/users/7c67911e2cf84f99a9e077e89cae272e/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:01.137216 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c4cca-024c-4930-b468-4bf6a8064c0b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:01.148630 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c4cca-024c-4930-b468-4bf6a8064c0b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:01.223547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:01.224152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:01.224426 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:01.224695 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:01.225327 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:01.226954 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:01.236795 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1288/2580] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Nov 2 07:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:41:01.241300 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758a503-3f89-4202-9d52-696cfd30125d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:01.253055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758a503-3f89-4202-9d52-696cfd30125d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:01.257587 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:01.258391 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:01.258673 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:01.258944 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] PATH_INFO: `/v3/projects/02104a370e6243009c39ebf1979dad00/users/7c67911e2cf84f99a9e077e89cae272e/roles/f5e3c85c776c4e7392fac5e1418c0bbc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:01.260825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02104a370e6243009c39ebf1979dad00, user_id=7c67911e2cf84f99a9e077e89cae272e, role_id=f5e3c85c776c4e7392fac5e1418c0bbc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:01.263458 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:01.273875 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1293/2581] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Nov 2 07:41:01 2022] PUT /identity/v3/projects/02104a370e6243009c39ebf1979dad00/users/7c67911e2cf84f99a9e077e89cae272e/roles/f5e3c85c776c4e7392fac5e1418c0bbc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:41:05.579521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:05.581470 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:05.709431 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:05.712272 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:05.785523 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abc9e8-3d5a-438a-b872-ac25dac78825 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:05.834263 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:05.877638 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f3b47-325c-4f11-8439-1c306708c1f3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:05.882790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:05.883447 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:05.884071 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:05.884609 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] PATH_INFO: `/v3/users/2ce997cd838648ddaff729500e87f9b8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:05.885790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce997cd838648ddaff729500e87f9b8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:05.887982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:05.892010 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:05.892982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:05.893323 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:05.894343 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:05.896706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:05.916699 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:05.918900 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:05.936236 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:05.954207 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1289/2582] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:06.020264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce997cd838648ddaff729500e87f9b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.021076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce997cd838648ddaff729500e87f9b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.046790 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce997cd838648ddaff729500e87f9b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.057854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce997cd838648ddaff729500e87f9b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.058698 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-70316ee3-8c19-4542-87d4-e3bb933b2b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce997cd838648ddaff729500e87f9b8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.076258 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1294/2583] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:41:05 2022] DELETE /identity/v3/users/2ce997cd838648ddaff729500e87f9b8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:41:06.311049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:06.333512 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2726d-17e4-468c-b2f1-113c931b2b9c service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:06.426336 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:06.428604 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:06.428896 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:06.429170 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] PATH_INFO: `/v3/projects/b34afcf6799f4d84ada9a295d1aea7c8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:06.431106 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34afcf6799f4d84ada9a295d1aea7c8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:06.433112 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:06.458854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535baa5-3b28-42a2-a9f3-d8f38dbde994 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:06.459214 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535baa5-3b28-42a2-a9f3-d8f38dbde994 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:06.459415 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535baa5-3b28-42a2-a9f3-d8f38dbde994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:06.519257 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1535baa5-3b28-42a2-a9f3-d8f38dbde994 None None] MFA Rules not processed for user `7c67911e2cf84f99a9e077e89cae272e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:06.521747 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1535baa5-3b28-42a2-a9f3-d8f38dbde994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:06.530309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34afcf6799f4d84ada9a295d1aea7c8', 'request_id': 'req-ccf71a83-d3ec-40e8-b8c9-e430a0380749'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.538593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34afcf6799f4d84ada9a295d1aea7c8', 'request_id': 'req-ccf71a83-d3ec-40e8-b8c9-e430a0380749'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.551299 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34afcf6799f4d84ada9a295d1aea7c8', 'request_id': 'req-ccf71a83-d3ec-40e8-b8c9-e430a0380749'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.575534 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] The token cache is being invalidate because project b34afcf6799f4d84ada9a295d1aea7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:41:06.575825 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.578826 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:41:06.584584 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1290/2584] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:41:06 2022] DELETE /identity/v3/projects/b34afcf6799f4d84ada9a295d1aea7c8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:41:06.685061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1295/2585] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:41:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:06.707150 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:06.712570 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:06.790521 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:06.839103 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf71a83-d3ec-40e8-b8c9-e430a0380749 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:06.848812 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:06.849535 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:06.849881 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:06.850288 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:06.852222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:06.938625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:06.940243 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:06.952469 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1291/2586] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:41:16.675789 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=56,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:16.946124 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=34 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:28.708291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2294b89-bc09-4709-871f-53f7ff40bbeb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:28.708813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2294b89-bc09-4709-871f-53f7ff40bbeb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:28.708813 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2294b89-bc09-4709-871f-53f7ff40bbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:28.732584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a2294b89-bc09-4709-871f-53f7ff40bbeb None None] MFA Rules not processed for user `f73df5ee33b14900bc8cbaa35275ac48`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:41:28.733792 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a2294b89-bc09-4709-871f-53f7ff40bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:28.745930 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1296/2587] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:41:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:41:30.051317 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:30.053068 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:30.125769 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9d366-4cca-4425-899b-d0aa7c8db618 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:30.131291 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:30.131886 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:30.132157 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:30.132425 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:30.134282 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:30.145948 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:30.147547 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:30.157706 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1292/2588] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:38.724394 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:40.153821 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=32 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:46.610076 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2294b89-bc09-4709-871f-53f7ff40bbeb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:46.611738 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a2294b89-bc09-4709-871f-53f7ff40bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:46.679382 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2294b89-bc09-4709-871f-53f7ff40bbeb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:46.684042 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:46.684649 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:46.684925 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:46.685190 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:46.695679 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:46.697115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:46.707211 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1297/2589] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:48.592776 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:48.605785 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc45fbe-f0af-431d-afbd-f92d9be5265d service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:48.611249 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:48.611875 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:48.612152 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:48.612421 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:48.614221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:48.627538 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:48.629101 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:48.640931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1293/2590] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:56.702748 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=26 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:58.637281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:53.843075 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:53.844663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:53.857297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707307-3e3d-4a1d-8911-389f64bb9bdd service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:53.861367 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d55ec4-fa61-42e6-8f8d-31b8a8736159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:53.861964 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d55ec4-fa61-42e6-8f8d-31b8a8736159 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:53.862299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d55ec4-fa61-42e6-8f8d-31b8a8736159 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:53.862574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d55ec4-fa61-42e6-8f8d-31b8a8736159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:53.864339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-57d55ec4-fa61-42e6-8f8d-31b8a8736159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:53.877829 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d55ec4-fa61-42e6-8f8d-31b8a8736159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:53.879281 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d55ec4-fa61-42e6-8f8d-31b8a8736159 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:53.891037 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1298/2591] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:42:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:42:55.113479 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:55.127808 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755b6f-ad48-4e7b-807b-0926a84db0c9 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:55.133634 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa51537-3e0f-495e-9e3b-f27c97b995c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:55.134259 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa51537-3e0f-495e-9e3b-f27c97b995c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:55.134593 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa51537-3e0f-495e-9e3b-f27c97b995c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:55.134962 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa51537-3e0f-495e-9e3b-f27c97b995c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:55.137616 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9fa51537-3e0f-495e-9e3b-f27c97b995c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:55.152438 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa51537-3e0f-495e-9e3b-f27c97b995c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:55.153894 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa51537-3e0f-495e-9e3b-f27c97b995c1 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:55.165430 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1294/2592] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:42:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:43:03.885506 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:43:05.160602 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:43:08.595605 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049119c9-3820-4614-8a0b-082c25b88bd6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:43:08.596200 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049119c9-3820-4614-8a0b-082c25b88bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:43:08.596478 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049119c9-3820-4614-8a0b-082c25b88bd6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:43:08.598143 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1299/2593] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:43:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:43:08.604917 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8f91-0afb-4657-af9b-106a870d9503 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:43:08.605483 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8f91-0afb-4657-af9b-106a870d9503 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:43:08.605675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8f91-0afb-4657-af9b-106a870d9503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:43:08.647004 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-03ca8f91-0afb-4657-af9b-106a870d9503 None None] MFA Rules not processed for user `c4ad0e5567bd452eadd14ed6ab1edc21`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:43:08.648234 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-03ca8f91-0afb-4657-af9b-106a870d9503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:43:08.654880 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1295/2594] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:43:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:43:08.668124 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049119c9-3820-4614-8a0b-082c25b88bd6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:43:08.679263 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049119c9-3820-4614-8a0b-082c25b88bd6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:43:08.683055 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a659e5-d88b-4f14-bf25-5b185857a4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:43:08.683660 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a659e5-d88b-4f14-bf25-5b185857a4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:43:08.683932 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a659e5-d88b-4f14-bf25-5b185857a4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:43:08.684339 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a659e5-d88b-4f14-bf25-5b185857a4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:43:08.691727 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a659e5-d88b-4f14-bf25-5b185857a4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:43:08.693179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a659e5-d88b-4f14-bf25-5b185857a4c3 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:43:08.701244 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1300/2595] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:43:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:43:08.786480 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ea08c-b5a5-4a24-9409-19018be4a2f3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:43:08.786774 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ea08c-b5a5-4a24-9409-19018be4a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:43:08.787049 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ea08c-b5a5-4a24-9409-19018be4a2f3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:43:08.788447 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1296/2596] 10.222.0.25 () {54 vars in 921 bytes} [Wed Nov 2 07:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:43:08.794164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e36c-f029-4f08-ad55-11ba0403c32b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:43:08.794450 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e36c-f029-4f08-ad55-11ba0403c32b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:43:08.794736 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e36c-f029-4f08-ad55-11ba0403c32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:43:08.835467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8193e36c-f029-4f08-ad55-11ba0403c32b None None] MFA Rules not processed for user `0722fc844689413796c658582ce00eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:43:08.836690 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8193e36c-f029-4f08-ad55-11ba0403c32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:43:08.848700 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1301/2597] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Nov 2 07:43:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:43:08.859800 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ea08c-b5a5-4a24-9409-19018be4a2f3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:43:08.872000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ea08c-b5a5-4a24-9409-19018be4a2f3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:43:08.876178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ea978-6f3c-4f9a-916e-e271cf87f3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:43:08.876768 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ea978-6f3c-4f9a-916e-e271cf87f3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:43:08.877041 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ea978-6f3c-4f9a-916e-e271cf87f3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:43:08.877309 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ea978-6f3c-4f9a-916e-e271cf87f3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:43:08.887968 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ea978-6f3c-4f9a-916e-e271cf87f3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:43:08.889404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ea978-6f3c-4f9a-916e-e271cf87f3f7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:43:08.900265 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1297/2598] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:43:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:43:18.832942 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=15 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:43:18.895738 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=15 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:44:06.592987 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193e36c-f029-4f08-ad55-11ba0403c32b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:44:06.607321 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193e36c-f029-4f08-ad55-11ba0403c32b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:44:06.613177 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:44:06.614547 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:44:06.615413 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:44:06.615800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:44:06.617996 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:06.630887 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:44:06.632326 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:44:06.641791 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1302/2599] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:44:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:44:07.528126 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39550e23-06fb-4a65-a030-3c36909e9575 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:44:07.528681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39550e23-06fb-4a65-a030-3c36909e9575 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:44:07.528681 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39550e23-06fb-4a65-a030-3c36909e9575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:44:07.550114 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-39550e23-06fb-4a65-a030-3c36909e9575 None None] MFA Rules not processed for user `f73df5ee33b14900bc8cbaa35275ac48`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:44:07.551329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-39550e23-06fb-4a65-a030-3c36909e9575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:07.559825 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1298/2600] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:44:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:44:16.637310 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:44:17.544757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:44:30.468574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:44:30.470408 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:30.487286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a85e2-c2bb-4986-99ce-ee8c9a523fbc service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:44:30.492945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:44:30.493555 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:44:30.493832 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:44:30.494129 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:44:30.496045 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:30.508521 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:44:30.510264 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:44:30.521373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1303/2601] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:44:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:44:40.516833 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:44:53.321740 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39550e23-06fb-4a65-a030-3c36909e9575 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:44:53.337582 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39550e23-06fb-4a65-a030-3c36909e9575 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:44:53.342404 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:44:53.343002 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:44:53.343275 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:44:53.343536 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:44:53.346205 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:53.406557 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:44:53.407986 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:44:53.417428 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1299/2602] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:44:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:44:59.539682 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:44:59.549333 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a0062-1dfd-4ea5-9eb5-e43db9bd8e49 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:44:59.552982 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:44:59.553573 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:44:59.553845 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:44:59.554145 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:44:59.555940 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:59.628820 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:44:59.630297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:44:59.640149 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1304/2603] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:45:02.541709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:45:02.543298 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:45:02.558304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41b1d-401d-4899-8d1c-7edb005ff2c7 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:45:02.564931 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:45:02.565601 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:45:02.565872 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:45:02.566173 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:45:02.568027 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:45:02.581304 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:45:02.582937 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:45:02.594222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1300/2604] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:45:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:45:05.887136 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:45:05.888775 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:45:05.901369 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92114dc8-398c-4fd9-b6d0-0c7a1c252d72 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:45:05.906938 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:45:05.907574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:45:05.907852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:45:05.908121 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] PATH_INFO: `/v3/users/ccdebe0f1a1646acbadf5cca7e97c7d2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:45:05.909345 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccdebe0f1a1646acbadf5cca7e97c7d2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:45:05.911317 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:45:05.961229 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccdebe0f1a1646acbadf5cca7e97c7d2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:05.961895 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccdebe0f1a1646acbadf5cca7e97c7d2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:05.976179 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccdebe0f1a1646acbadf5cca7e97c7d2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:05.984246 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccdebe0f1a1646acbadf5cca7e97c7d2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:05.985009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccdebe0f1a1646acbadf5cca7e97c7d2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.001923 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1305/2605] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:45:05 2022] DELETE /identity/v3/users/ccdebe0f1a1646acbadf5cca7e97c7d2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:45:06.189348 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:45:06.202833 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa99288-9d0f-44bf-b8ca-59b28cf04e89 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:45:06.265910 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:45:06.266566 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:45:06.266842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:45:06.267108 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] PATH_INFO: `/v3/projects/59ab1e9b63ae4a7e851eb1de51d64aef` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:45:06.268219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ab1e9b63ae4a7e851eb1de51d64aef)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:45:06.270175 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:45:06.323165 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ab1e9b63ae4a7e851eb1de51d64aef', 'request_id': 'req-ad56759c-046c-45d6-a774-c92f2d09352c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.329181 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ab1e9b63ae4a7e851eb1de51d64aef', 'request_id': 'req-ad56759c-046c-45d6-a774-c92f2d09352c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.340070 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ab1e9b63ae4a7e851eb1de51d64aef', 'request_id': 'req-ad56759c-046c-45d6-a774-c92f2d09352c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.353842 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] The token cache is being invalidate because project 59ab1e9b63ae4a7e851eb1de51d64aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:45:06.354121 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.354757 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.356221 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1301/2606] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:45:06 2022] DELETE /identity/v3/projects/59ab1e9b63ae4a7e851eb1de51d64aef => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:45:06.360421 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:45:06.361827 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:45:06.375080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57c1e3-8a51-496d-8321-536d9ecb28d0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:45:06.440899 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:45:06.441860 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:45:06.442342 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:45:06.442842 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] PATH_INFO: `/v3/users/02a570a4c3a34b22ab4d9a4be1bdc1fc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:45:06.444762 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a570a4c3a34b22ab4d9a4be1bdc1fc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:45:06.446707 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:45:06.452348 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:45:06.506523 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a570a4c3a34b22ab4d9a4be1bdc1fc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.507496 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a570a4c3a34b22ab4d9a4be1bdc1fc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.520564 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a570a4c3a34b22ab4d9a4be1bdc1fc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.528373 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a570a4c3a34b22ab4d9a4be1bdc1fc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.529052 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a570a4c3a34b22ab4d9a4be1bdc1fc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.544164 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1306/2607] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:45:06 2022] DELETE /identity/v3/users/02a570a4c3a34b22ab4d9a4be1bdc1fc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:45:06.726493 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:45:06.728195 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:45:06.744549 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56759c-046c-45d6-a774-c92f2d09352c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:45:06.758789 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:45:06.805260 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:45:06.805914 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:45:06.806233 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:45:06.806508 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] PATH_INFO: `/v3/projects/6023341c866c41f48f9210997de52a39` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:45:06.807709 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6023341c866c41f48f9210997de52a39)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:45:06.810149 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:45:06.856409 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6023341c866c41f48f9210997de52a39', 'request_id': 'req-292678f7-925f-4be5-badd-57b051d0f1e8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.862675 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6023341c866c41f48f9210997de52a39', 'request_id': 'req-292678f7-925f-4be5-badd-57b051d0f1e8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.871413 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6023341c866c41f48f9210997de52a39', 'request_id': 'req-292678f7-925f-4be5-badd-57b051d0f1e8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.887817 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] The token cache is being invalidate because project 6023341c866c41f48f9210997de52a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:45:06.888213 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.889254 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-292678f7-925f-4be5-badd-57b051d0f1e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:45:06.891723 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1302/2608] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:45:06 2022] DELETE /identity/v3/projects/6023341c866c41f48f9210997de52a39 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:45:16.540884 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:45:16.882514 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:46:19.306618 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:46:19.309241 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:19.406656 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b8efa-9409-411f-ad97-a362342161cc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:46:19.466548 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17ba458c5004646b5e21cabcb54fd20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:46:19.467163 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:46:19.467443 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:46:19.467717 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:46:19.469661 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:19.548811 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:46:19.550243 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:46:19.560480 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1307/2609] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:46:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:46:28.615651 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebe7c0-a0e6-40a2-aac3-626208613dbc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:46:28.616226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebe7c0-a0e6-40a2-aac3-626208613dbc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:46:28.616226 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebe7c0-a0e6-40a2-aac3-626208613dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:46:28.636989 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7aebe7c0-a0e6-40a2-aac3-626208613dbc None None] MFA Rules not processed for user `f73df5ee33b14900bc8cbaa35275ac48`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:46:28.638222 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7aebe7c0-a0e6-40a2-aac3-626208613dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:28.645363 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1303/2610] 10.222.0.25 () {58 vars in 958 bytes} [Wed Nov 2 07:46:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:46:29.001252 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:46:29.003662 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:29.044665 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:46:29.073546 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716cab0-ceec-40ef-a61d-5cedef882943 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:46:29.077678 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:46:29.078322 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:46:29.078601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:46:29.078950 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:46:29.080904 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:29.091467 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:46:29.092892 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:46:29.101670 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1308/2611] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:46:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:46:29.879197 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebe7c0-a0e6-40a2-aac3-626208613dbc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:46:29.880635 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7aebe7c0-a0e6-40a2-aac3-626208613dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:29.984493 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebe7c0-a0e6-40a2-aac3-626208613dbc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:46:29.990219 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:46:29.990823 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:46:29.991096 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:46:29.991365 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:46:29.993513 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:30.005490 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:46:30.007089 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:46:30.017192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1304/2612] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:46:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:46:30.529291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:46:30.530937 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:30.546537 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dc541-1f5c-4e43-b4f2-004463c47bb8 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:46:30.551993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:46:30.552592 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:46:30.552854 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:46:30.553131 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:46:30.555061 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:30.567487 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:46:30.569104 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:46:30.578953 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1309/2613] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:46:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:46:40.013167 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=47 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:46:40.574765 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=19 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:46:44.737874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:46:44.739501 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:44.825329 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a514b5-bb88-49d6-9232-0709796bb0ec service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:46:44.831055 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:46:44.831666 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:46:44.831946 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:46:44.832218 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:46:44.833982 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:44.846708 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:46:44.848281 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:46:44.858932 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1305/2614] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:46:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:46:48.098984 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:46:48.114173 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc95e7-6699-4cab-9510-62c76117587d service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:46:48.121148 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:46:48.122473 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:46:48.122746 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:46:48.123003 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:46:48.125064 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:46:48.200825 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:46:48.202286 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:46:48.212406 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1310/2615] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:46:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:46:54.854639 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=37 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:46:58.207753 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=30 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:48:43.704541 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:48:43.725783 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0e79d-3365-4c62-adbd-ce267a8b648f service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:48:43.729752 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:48:43.730474 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:48:43.730830 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:48:43.731178 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:48:43.733255 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:43.762431 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:48:43.763883 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:48:43.784915 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1306/2616] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:48:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:48:44.991216 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:48:45.014744 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4987fa-4d19-405b-9c2f-d45b6da5508e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:48:45.018723 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:48:45.019323 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:48:45.019593 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:48:45.019863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:48:45.021755 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:45.037009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:48:45.038768 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:48:45.050759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1311/2617] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:48:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:48:50.576997 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:48:50.578573 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:50.595746 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d363-4e41-433d-8664-d1413a00b04f service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:48:50.603625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0722fc844689413796c658582ce00eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:48:50.603625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:48:50.603625 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:48:50.604140 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:48:50.606345 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:50.620156 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:48:50.621874 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:48:50.635133 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1307/2618] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:48:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:48:55.045335 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:48:56.197144 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:48:56.206287 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72058d-5644-46c4-b095-b989073954ef service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:48:56.210713 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4ad0e5567bd452eadd14ed6ab1edc21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:48:56.211312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:48:56.211584 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:48:56.211850 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:48:56.213852 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:56.299181 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:48:56.300624 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:48:56.319663 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1312/2619] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:48:59.237350 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:48:59.250923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b9696-3069-4ef3-985a-ed01a602a73b service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:48:59.256467 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:48:59.257192 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:48:59.257526 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:48:59.257818 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:48:59.259727 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:59.274458 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:48:59.276435 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:48:59.288099 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1308/2620] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:48:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:49:02.591299 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:49:02.592841 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:49:02.605518 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4259f8-cc8f-406e-a9b2-91df7fe6fe12 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:49:02.610612 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:49:02.611251 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:49:02.611527 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:49:02.611800 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] PATH_INFO: `/v3/users/f73df5ee33b14900bc8cbaa35275ac48` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:49:02.613009 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f73df5ee33b14900bc8cbaa35275ac48)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:49:02.614972 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:49:02.666535 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f73df5ee33b14900bc8cbaa35275ac48'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:02.667280 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f73df5ee33b14900bc8cbaa35275ac48'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:02.679601 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f73df5ee33b14900bc8cbaa35275ac48'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:02.687291 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f73df5ee33b14900bc8cbaa35275ac48'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:02.687993 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f73df5ee33b14900bc8cbaa35275ac48'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:02.703222 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1313/2621] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:49:02 2022] DELETE /identity/v3/users/f73df5ee33b14900bc8cbaa35275ac48 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:49:02.889072 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:49:02.900925 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ed4d-9db1-45a8-b4f5-39d1e84d5ea5 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:49:02.960993 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:49:02.961237 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:49:02.961511 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:49:02.961777 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] PATH_INFO: `/v3/projects/5d00af8d4afc4004aa95bd16a703c148` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:49:02.963000 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d00af8d4afc4004aa95bd16a703c148)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:49:02.964973 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:49:03.012758 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d00af8d4afc4004aa95bd16a703c148', 'request_id': 'req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.018697 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d00af8d4afc4004aa95bd16a703c148', 'request_id': 'req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.027977 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d00af8d4afc4004aa95bd16a703c148', 'request_id': 'req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.038641 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] The token cache is being invalidate because project 5d00af8d4afc4004aa95bd16a703c148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:49:03.038923 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.039530 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.040980 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1309/2622] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:49:02 2022] DELETE /identity/v3/projects/5d00af8d4afc4004aa95bd16a703c148 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:49:03.045331 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:49:03.046863 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:49:03.059945 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cde2a-29a2-4f7d-a773-2d58a4d3a825 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:49:03.095614 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:49:03.122979 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:49:03.123606 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:49:03.123889 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:49:03.124163 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] PATH_INFO: `/v3/users/7c67911e2cf84f99a9e077e89cae272e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:49:03.125312 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c67911e2cf84f99a9e077e89cae272e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:49:03.127293 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:49:03.181096 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c67911e2cf84f99a9e077e89cae272e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.181759 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c67911e2cf84f99a9e077e89cae272e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.194115 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c67911e2cf84f99a9e077e89cae272e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.201418 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c67911e2cf84f99a9e077e89cae272e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.202080 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c67911e2cf84f99a9e077e89cae272e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.217361 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1314/2623] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Nov 2 07:49:03 2022] DELETE /identity/v3/users/7c67911e2cf84f99a9e077e89cae272e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:49:03.403001 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:49:03.404638 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:49:03.418685 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2cb63-71b8-4cc9-bed5-2f4285a39cdc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:49:03.439064 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:49:03.479206 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '085301de455d4179ae28198d5b0bd385', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e25fb0b05a884f4d989636700db0537f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:49:03.479827 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:49:03.480128 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:49:03.480397 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] PATH_INFO: `/v3/projects/02104a370e6243009c39ebf1979dad00` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:49:03.481598 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02104a370e6243009c39ebf1979dad00)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:49:03.483506 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:49:03.531214 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02104a370e6243009c39ebf1979dad00', 'request_id': 'req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.537544 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02104a370e6243009c39ebf1979dad00', 'request_id': 'req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.547043 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02104a370e6243009c39ebf1979dad00', 'request_id': 'req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.558448 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] The token cache is being invalidate because project 02104a370e6243009c39ebf1979dad00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:49:03.558731 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.559310 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-60fa8263-ee4f-4f4e-84e9-4e4a7526ee29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:49:03.560824 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1310/2624] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Nov 2 07:49:03 2022] DELETE /identity/v3/projects/02104a370e6243009c39ebf1979dad00 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:49:13.213657 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:49:13.555509 dv-cinder-778193-30-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:52:13.866866 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:52:13.868449 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:52:13.946574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72ed10-430c-449b-8933-7455a0fdd24c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:52:14.011697 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6832a-2685-419f-a929-9a13eeceac22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db7ab631609f49249a010b1a8b03e002', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c271e6d0d23647089c07648d4c38193d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:52:14.012297 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6832a-2685-419f-a929-9a13eeceac22 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:52:14.012574 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6832a-2685-419f-a929-9a13eeceac22 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:52:14.012847 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6832a-2685-419f-a929-9a13eeceac22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:52:14.014902 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-59a6832a-2685-419f-a929-9a13eeceac22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:52:14.027126 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6832a-2685-419f-a929-9a13eeceac22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:52:14.028712 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6832a-2685-419f-a929-9a13eeceac22 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:52:14.038839 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1315/2625] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Nov 2 07:52:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:52:24.034784 dv-cinder-778193-30-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=58 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}